Question for training time

[Egkang-Luis]

Thanks for release codes.

I would like to train your Burst SR model using released codes but I want to ask something before training.

In the paper, it seems the model trained with 4 RTX6000 GPUs.

Could you share how long the train took?

Thanks.
Luis Kang.

[akshaydudhane16]

For synthetic burst SR, we train the proposed Burstormer for 300 epochs with 4 RTX6000 GPUs, and it takes roughly around 5-6 days. To fine-tune this model for real burst SR it takes around 8 hours.

[Egkang-Luis]

For synthetic burst SR, we train the proposed Burstormer for 300 epochs with 4 RTX6000 GPUs, and it takes roughly around 5-6 days. To fine-tune this model for real burst SR it takes around 8 hours.

Thanks for reply. Actually, my GPU is 3090 and I am training BIPNet using 4GPUs. Currently, I am training around 5days but still 51 epochs.
Could you let me know which model is faster between Bustomer and BIPNet?

Note that. To fair comparison with my model, I just changed precision from 16 to 32 and deterministic from True to False.

[akshaydudhane16]

Burstormer is faster than BIPNet. In our settings, we keep precision 16 and deterministic True.
In released codes, each GPU serves a single burst (so 4 GPUs combinely make batch size=4). So, to improve training time, you can implement the network architecture such that you can increase batch size.

[Egkang-Luis]

Burstormer is faster than BIPNet. In our settings, we keep precision 16 and deterministic True.
In released codes, each GPU serves a single burst (so 4 GPUs combinely make batch size=4). So, to improve training time, you can implement the network architecture such that you can increase batch size.

After getting your comment, I started to train Burstomer. I didn't change the setting that you provided. But the training time per epoch was 2h 10m. I can expect whole training time for 300 epochs 27 days, it was fully different that you comments.

I used training set Zurich RAW to RGB Dataset(22 GB)

Could you advise to get same training time that you mentioned? For example, save dataset on harddisk after applying preprocessing and augment, or use the other dataset such as Cannon RGB Images(5.5GB) that was commented in zurich_raw2rgb_dataset.py.

[Egkang-Luis]

One more question. I download pre-trained model that you provided for Synthetic(Track 1)

When I load weight and apply it to model, a lot of error were occurred.

I didn't change any code. Could you let me know do I need change code or could you check provided pre-trained model was correct?

Below is error code.

===========================================================================================
Missing key(s) in state_dict: "back_projection1.feat_fusion.0.weight", "back_projection1.feat_fusion.0.bias", "back_projection1.feat_expand.0.weight", "back_projection1.feat_expand.0.bias", "back_projection2.feat_fusion.0.weight", "back_projection2.feat_fusion.0.bias", "back_projection2.feat_expand.0.weight", "back_projection2.feat_expand.0.bias".
Unexpected key(s) in state_dict: "back_projection1.diff_fusion.weight", "back_projection1.feat_fusion.weight", "back_projection1.feat_expand.weight", "back_projection2.diff_fusion.weight", "back_projection2.feat_fusion.weight", "back_projection2.feat_expand.weight".
size mismatch for align.alignment0.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.alignment0.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.alignment0.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.alignment0.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.alignment0.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.alignment0.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.alignment0.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.alignment1.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.alignment1.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.alignment1.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.alignment1.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.alignment1.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.alignment1.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.alignment2.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.alignment2.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.alignment2.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.alignment2.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.alignment2.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.alignment2.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).

[ChongWang1024]

One more question. I download pre-trained model that you provided for Synthetic(Track 1)

When I load weight and apply it to model, a lot of error were occurred.

I didn't change any code. Could you let me know do I need change code or could you check provided pre-trained model was correct?

Below is error code.

=========================================================================================== Missing key(s) in state_dict: "back_projection1.feat_fusion.0.weight", "back_projection1.feat_fusion.0.bias", "back_projection1.feat_expand.0.weight", "back_projection1.feat_expand.0.bias", "back_projection2.feat_fusion.0.weight", "back_projection2.feat_fusion.0.bias", "back_projection2.feat_expand.0.weight", "back_projection2.feat_expand.0.bias". Unexpected key(s) in state_dict: "back_projection1.diff_fusion.weight", "back_projection1.feat_fusion.weight", "back_projection1.feat_expand.weight", "back_projection2.diff_fusion.weight", "back_projection2.feat_fusion.weight", "back_projection2.feat_expand.weight". size mismatch for align.alignment0.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.alignment0.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.alignment0.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.alignment0.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.alignment0.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.alignment0.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.alignment0.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.alignment1.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.alignment1.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.alignment1.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.alignment1.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.alignment1.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.alignment1.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.alignment2.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.alignment2.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.alignment2.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.alignment2.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.alignment2.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.alignment2.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).

Hi Luis, I got the same issue on Track_1_evaluation.py, have you solved it?

[Egkang-Luis]

Hi Luis, I got the same issue on Track_1_evaluation.py, have you solved it?

Hi ChongWang.

I can't solve above issue. DId you solve it?