Add informative logs in historical_forecasts() to display train and prediction TimeSeries characteristics #2945
Description
Is your feature request related to a current problem? Please describe.
When running historical_forecasts() for evaluation, I find really usefull to know the characteristics of the input TimeSeries (or list of TimeSeries) for each step of training and prediction. Having this visibility can make debugging or understanding evaluation behavior easier, especially when working with multiple series, changing from local to global models with apply_globally parameter, retraining with a callable, uneven frequencies, or different metadata (I'm really used to having a {"series_id": "..:"} as metadata in my TimeSeries objects).
Describe proposed solution
Add informative logs within historical_forecasts() via the verbosity flag, that display key characteristics of the input TimeSeries objects being used for training and prediction.
For example, these logs could summarize:
- Number of series in the input (if a list)
- Length, start and end timestamps
- Frequency
- Metadata (if available)
- Show if it's a TimeSeries or List[TimeSeries] to know if local or global training.
A minimal prototype I've been using is the following function:
def get_series_information(ts: TimeSeries | list[TimeSeries]) -> str:
if isinstance(ts, TimeSeries):
metadata_values = list(ts.metadata.values()) if ts.has_metadata else None
series_info = f"TimeSeries: len={len(ts)} - start={ts.start_time()} - end={ts.end_time()} - freq={ts.freq}"
if metadata_values is not None:
series_info += f" - metadata={metadata_values}"
return series_info
elif isinstance(ts, list) and all(isinstance(s, TimeSeries) for s in ts):
metadata_values = [list(s.metadata.values()) if s.has_metadata else None for s in ts]
total_len = sum(len(s) for s in ts)
min_start = min(s.start_time() for s in ts)
max_end = max(s.end_time() for s in ts)
freqs = set(s.freq for s in ts)
series_info = f"List[TimeSeries]: n_series={len(ts)} - total_len={total_len} - min_start={min_start} - max_end={max_end} - freq={freqs}"
if any(metadata_values):
series_info += f" - metadata={metadata_values}"
return series_info
else:
return "Unknown type"I then included a simple print for this function call in the historical_forecast method in the forecasting_model.py module in darts/models/forecasting.
# line 1208
if apply_retrain:
# fit a new instance of the model
model = model.untrained_model()
print(f"Training model on: {get_series_information(train_series_tf)}")
model._fit_wrapper(
series=train_series_tf,
past_covariates=past_covariates_tf,
future_covariates=future_covariates_tf,
sample_weight=sample_weight_tf,
val_series=val_series_tf,
**fit_kwargs,
)
# line 1222
# forecast:
# - local hfc returns a single TimeSeries
# - global hfc returns a list of TimeSeries
forecast_tf = model._predict_wrapper(
n=forecast_horizon,
series=pred_series_tf,
past_covariates=past_covariates_tf,
future_covariates=future_covariates_tf,
num_samples=num_samples,
predict_likelihood_parameters=predict_likelihood_parameters,
show_warnings=show_predict_warnings,
random_state=random_state,
**predict_kwargs,
)
print(f"Forecasting: {get_series_information(forecast_tf)}")This probably doesn't account for other historical_forecast implementations such as the optimized one. This is how it would look like in the Energy Dataset (subset) for global training:
Training on: List[TimeSeries]: len=6 - Series: total_len=4032 - min_start=2012-11-01 00:00:00 - max_end=2012-11-07 23:45:00 - metadata=[MT_031 MT_135 MT_164 MT_182 MT_295 MT_362]
Forecasting: List[TimeSeries]: len=6 - Series: total_len=72 - min_start=2012-11-08 00:00:00 - max_end=2012-11-08 02:45:00 - metadata=[MT_031 MT_135 MT_164 MT_182 MT_295 MT_362]
Forecasting: List[TimeSeries]: len=6 - Series: total_len=72 - min_start=2012-11-09 00:00:00 - max_end=2012-11-09 02:45:00 - metadata=[MT_031 MT_135 MT_164 MT_182 MT_295 MT_362]
Forecasting: List[TimeSeries]: len=6 - Series: total_len=72 - min_start=2012-11-10 00:00:00 - max_end=2012-11-10 02:45:00 - metadata=[MT_031 MT_135 MT_164 MT_182 MT_295 MT_362]
In the case of local training, we would have 6 times each log, with TimeSeries object with len 4032 as training input.
Describe potential alternatives
The approach above is just a simple implementation I'm currently using but it could likely be implemented in a cleaner or more integrated way within Darts’ logging system. Still, adding this kind of diagnostic information would be a great quality-of-life improvement for users evaluating time series models.