Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] TimeSeriesImputerTransform doesn't work in pipeline.backtest if there is NaN in prediction part #108

Open
1 task done
ostreech1997 opened this issue Oct 11, 2023 · 0 comments
Labels
bug Something isn't working

Comments

@ostreech1997
Copy link
Collaborator

🐛 Bug Report

TimeSeriesImputerTransform doesn't work in pipeline.backtest if there is NaN in prediction part.

Expected behavior

Should we change this behaviour?

How To Reproduce

Script:

from loguru import logger


def create_example_ts():
    import numpy as np
    import pandas as pd
    from etna.datasets import TSDataset

    target = np.random.normal(size=(20, 10))
    segments = [f"seg_{i}" for i in range(10)]
    df = pd.DataFrame(target, columns=segments)
    timestamp = pd.date_range("2020-01-01", periods=20, freq="D")
    df["timestamp"] = timestamp
    df.iloc[-2, 2] = np.nan
    df = df.melt("timestamp", var_name="segment", value_name="target")
    ts = TSDataset(TSDataset.to_dataset(df), freq="D")
    return ts


def pipeline_backtest():
    from etna.transforms import TimeSeriesImputerTransform
    from etna.pipeline import Pipeline
    from etna.models import ProphetModel
    from etna.metrics import MAE

    # Robust pipeline
    ts = create_example_ts()
    ts.fit_transform(transforms=[TimeSeriesImputerTransform(in_column="target", strategy="constant"),])
    pipeline = Pipeline(
        ProphetModel(weekly_seasonality=True,),
        transforms=[],
        horizon=2
    )
    _ = pipeline.backtest(
        ts=ts, metrics=[MAE()], n_folds=3, stride=1
    )
    logger.info("Robust pipeline is done!")

    # Fail pipeline
    ts = create_example_ts()
    pipeline = Pipeline(
        ProphetModel(weekly_seasonality=True, ),
        transforms=[TimeSeriesImputerTransform(in_column="target", strategy="constant"),],
        horizon=2
    )
    _ = pipeline.backtest(
        ts=ts, metrics=[MAE()], n_folds=3, stride=1
    )
    logger.info("Fail pipeline is done!")


def main() -> None:
    pipeline_backtest()


if __name__ == "__main__":
    main()

Environment

No response

Additional context

No response

Checklist

  • Bug appears at the latest library version
@ostreech1997 ostreech1997 added the bug Something isn't working label Oct 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant