regarding use of loss function sklearn make_circles classification problem

[harivan86]

Hello everyone,

I replaced
loss_fn = nn.BCEWithLogitsLoss()
with this below line
loss_fn = nn.BCELoss()

and in the training loop:
I replaced
loss = loss_fn(y_logits, y_train)
with this below line
loss = loss_fn(y_pred, y_train)

Do note that
y_pred = torch.round(torch.sigmoid(y_logits))
which is the requirement of BCELoss as per pytorch documentation
https://pytorch.org/docs/stable/generated/torch.nn.BCELoss.html

With these changes from what Daniel sir taught in the online course lecture of Pytorch in the lecture
"Making Predictions with and Evaluating Our First Non-Linear Model"

I ran the code, but the loss is not decreasing. Can Daniel sir or somebody help.
Attached is the python script in .txt format
concentric_circle_dots_classification.txt

Thanks,
Hari

[LuluW8071]

@harivan86 seems like you forgot to set the model to train mode on ur training loop.

Corrected code:

for epoch in range(epochs):
    model.train()   # Set your model (model_name) to train mode
    
    # 1. Forward pass
    y_logits = model_3(X_train).squeeze()
    y_pred = torch.round(torch.sigmoid(y_logits)) # logits -> prediction probabilities -> prediction labels

Can you run you code once again with above changes.

[harivan86]

Hi @LuluW8071 ,
I have now run the code including the model_3.train() that I earlier missed out, but the result with nn.BCELoss() is still the same (loss not converging and remaining same on each epoch)

Thanks,
Hari

[LuluW8071]

@harivan86 You were passing the rounded logit values (after applying sigmoid) to the loss function causing the loss (BCE Loss) not to decrease.

Note

Use y_pred = torch.round(torch.sigmoid(y_logits)) to calculate accuracy. For BCELoss, as stated in the PyTorch documentation, pass the sigmoid logits only rather than the rounded sigmoid logit values.
https://pytorch.org/docs/stable/generated/torch.nn.BCELoss.html

Here’s the corrected code:

for epoch in range(epochs):
    model_3.train()
    # 1. Forward pass
    y_logits = model_3(X_train).squeeze()
    y_pred = torch.sigmoid(y_logits)  # Convert logits to prediction probabilities

    # 2. Calculate loss and accuracy
    loss = loss_fn(y_pred, y_train)  # BCELoss requires non-rounded sigmoid logits
    acc = accuracy_fn(y_true=y_train,
                      y_pred=torch.round(y_pred))  # Round sigmoid logits for accuracy

Apply the same operation to the test loop it should work.

[harivan86]

Thank you @LuluW8071 for your valuable time in clarifying this.