Questions about experiment results.

[ghost]

I trained about 50000 iters using 2080Ti with batch size as 2, and I found that the evaluation results are quite unstable. The AP50 fluctuated around 41 and reach a maximum of 43.5. I wanna ask you how to judge the convergence of the model and select the results to report.
Thanks a lot.

[wymanCV]

Hi, thanks for your attention.

First of all, the instability is common due to the adversarial training, which is the same as almost all existing DAOD frameworks, e.g., EPM (our baseline), DAF, and the latest work VISGA. As far as I know, currently, no framework can guarantee stable training and completed convergence with GRL-based adversarial learning (without using Cyclegan to generate interpolated data). So, you may notice that almost all current works rely on the model selection for the benchmark comparison. If you can address the GRL-based instability, it will be a big step for this field.

Actually, longer training, two-stage training, and reducing the learning rate will make the training more stable but won't break the maximum results a lot. So, the common and easy practice is to train longer, e.g., AT uses 360000 iterations w. bs=16. Moreover, you may find that our uploaded model should have higher AP (not AP50) than your reproduced one because we report our results (bs=4) with longer training and two-stage training when the model has a tolerable fluctuation around 43. You can load it as the initialization and train further to have a look.

Besides, some interesting discussion (in Chinese) about the convergence of GRL-based DA may help you understand this common phenomenon better.