Assessing convergence of the neural networkw with sbi

[paarth-dudani]

Can I get some suggestions as to how I can evaluate whether the neural network training has converged?

One idea that comes to my mind it to output the loss function and view how it changes with time. How do I do that in the case of NPE_C, NRE_B and NLE inference estimators?

[michaeldeistler]

Hi there! The sbi toolbox automatically performs early stopping. It will train until the validation loss does longer decrease for 20 epochs.

You can also plot the loss with:

from sbi.analysis import plot_summary

inference = NPE(prior)
_ = inference.append_simulations(theta, x).train()

plot_summary(inference)

See also here

[paarth-dudani]

Hi
Thanks for the answer!

Could you also tell me how I can output the training loss and other inference parameters?

[michaeldeistler]

Hi! You can do:

plot_summary(inference, tags=["training_loss", "validation_loss"])

What do you mean by "other inference parameters"?

[paarth-dudani]

Thanks for the suggestion!

I meant for instance, weights at different number of epochs?

[michaeldeistler]

The only things that are tracked are: ['best_validation_loss', 'epoch_durations_sec', 'epochs_trained', 'training_loss', 'validation_loss']

Weights at different number of epochs are not tracked. If you really need this much control over the training loop, then sbi offers the option to fully write the training loop yourself, see this tutorial.

[paarth-dudani]

Alright! Thanks for the suggestion! I will try it out