Training

[Luke2642]

I was just wondering if you think your two interesting innovations RAU-Net and MSW-MSA would also have any benefit during fine tuning? Do you think the model would train faster or generalise better, or need only to be trained on smaller crops and downsampled images rather than high resolution images? If so that could be quite a boost to training!

I skimmed the paper but it was focused on the improvement during inference.

[ShenZhang-Shin]

Both RAU-Net and MSW-MSA can speed up the forward speed, thereby enabling faster training.
If you have datasets with a resolution of 1024x1024, I believe that using RAU-Net to fine-tune a model trained on 512x512 resolution could boost convergence. This is because, after applying RAU-Net, the size of the features within the deep blocks of network remains consistent with those of a 512x512 resolution.
High-resolution images are precious and rare. I suggest training models using high-resolution images, and then generating higher-resolution images with HiDiffusion.