Clarification on Early Stopping Behavior with Multiple eval_set in LightGBM

[JuseTiZ]

Hello LightGBM Team,

I have a question regarding the early stopping mechanism when multiple evaluation datasets (eval_set) are specified in the fit method.

Example Code:

callbacks = [early_stopping(stopping_rounds=75)]
light.fit(
    X_train,
    y_train,
    eval_names=['valid1', 'valid2'],
    eval_set=[(X_valid1, y_valid1), (X_valid2, y_valid2)],
    callbacks=callbacks
)

Documentation:

Requires at least one validation data and one metric. If there’s more than one, will check all of them. But the training data is ignored anyway.
https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.early_stopping.html

I have a question regarding the phrase "check all of them" in the context of LightGBM's early stopping. Does this mean that LightGBM merges the evaluation datasets and makes a decision based on the combined performance? Or does it monitor each dataset individually and trigger early stopping if any one of them shows no improvement?

[jameslamb]

Thanks for using LightGBM.

In the Python package, each evaluation dataset is monitored individually, and early stopping can be triggered based on any one of them. You may want to subscribe to notifications #6424 and #6360 to keep up with potential changes to allow finer control of that behavior.

[JuseTiZ]

Clear explanation, thank you.