[ENH] Does TimeSeriesDataSet supports passing in file list?

[Neronjust2017]

class pytorch_forecasting.data.timeseries._timeseries.TimeSeriesDataSet(data: DataFrame, time_idx: str, target: str | list[str], group_ids: list[str], weight: str | None = None, ...

It seems that TimestamperiesDataset only supports dataframes, but dataframes may be very large, leading to OOM issues. Does it support passing in csv or parquet lists, etc., to dynamically load data? Other memory-saving methods are also recommended. Thanks!

[phoeenniixx]

As far as I know, TimeSeriesDataset accepts dataframes only. You'll have to handle the OOM issues manually unfortunately, maybe using chuking? But I am not entirely sure how would you stream the data from disk to TimeSeriesDataset as .from_dataset() uses the data from the memory only.

We are trying to solve this shortcoming for v2.0 release. Comments are appreciated here - #1736