section 02: My problem is, I am not getting how optimization and back propogation are implemented via codes.

[VivekChimman]

epochs=100
for epoch in epochs:
      ### Training 
      model_0.train() 
      y_pred=model_0(X_train)
      loss=loss_fn(y_pred, y_train)
      optimizer.zero_grad()
      loss.backward
      optimizer.step()
      
### Testing
      model_0.eval()
      with torh.inference_mode():
            test_pred=model_0(X_test)
            test_loss=loss_fn(test_pred, y_test)
      
      print(f"Epoch: {epoch} | loss: {loss} | Test loss: {test_loss}")

Here in training loop optimizer.zero_grad(), loss.backward(), optimizer.step() look like not connected with each other then how they are helping model to get trained on the data via optimization and back propogation? What's happening there?

[LuluW8071]

Learn about gradient descent from linear regression and maths behind it. You will get gist of what is happening behind. Then watch Andrej Karpathy micrograd vid on YouTube.

[VivekChimman]

I know that. I am asking about codes; sequentially.

[LuluW8071]

Just look at the library codes or source code in pytorch
Search torch.optim.optimizer source code