Forecasting time series
Vegan haruspicy
2015-06-16 — 2026-10-20
In Which Tsibble and Fable Are Employed, Recursive Identification Is Reconciled With Foundation Models Like Chronos-2, and M4’s 100,000 Series Are Compared Against the Fev-Bench Leaderboard’s 100 Tasks.
On this page
Time series prediction niceties, where what we need to predict is the future. Filed under forecasting because in machine learning terminology, prediction is a general term that does not necessarily imply extrapolation into the future.
🚧TODO🚧 handball to Rob Hyndman.
1 Recursive estimation
See recursive identification for the generic theory of learning under the distribution shift induced by a moving parameter vector.
2 Model selection
Rob Hyndman explains how to cross-validate time series models that use only the lagged observations, and Bergmeir, Hyndman, and Koo (2018) show when ordinary \(K\)-fold cross-validation is valid for purely autoregressive models. Cosma Shalizi mentions the sample splitting problem for time series for post-selection inference and has supervised students to do some work with it, notably (Lunde 2019).
For a different emphasis on the same problem, consider statistical learning theory, or model ensembles in times series.
Or: would we like to try model mixing in this setting? See ensemble methods.
3 Calibration of probabilistic forecasts
See calibration.
4 Training data intake
There is a common pattern with training time series models that they each predict the next observation from the previous observations, which is not how a classic data loader works in machine learning. The time at which the future observations are evaluated is the horizon and the ones used to make that prediction are the history. For patterns to handle this in neural networks in particular, see Recurrent neural networks.
5 Software
Not comprehensive, just noting some useful time series forecasting models/packages as I encounter them. Peter Cotton attempts to collate Popular Python Time Series Packages.
5.1 Tidyverse time series analysis and forecasting packages
A good first stop. The accompanying textbook is R. J. Hyndman and Athanasopoulos (2021), free online.
You can find a presentation on these tools by Rob Hyndman.
- tsibble: Tidy Temporal Data Frames and Tools [CRAN] (E. Wang, Cook, and Hyndman 2020a)
- tsibbledata: Example datasets for tsibble [CRAN]
- feasts: Feature Extraction And Statistics for Time Series [CRAN]
- fable: Forecasting Models for Tidy Time Series [CRAN]
- sugrrants: Supporting Graphs for Analysing Time Series. Tools for plotting temporal data using the tidyverse and grammar of graphics framework. [CRAN] (E. Wang, Cook, and Hyndman 2020b)
- gravitas: Explore Probability Distributions for Bivariate Temporal Granularities. [CRAN] (Gupta et al. 2021)
5.2 scikit-learn style: Nixtla, sktime, darts
Rob J Hyndman, in Python implementations of time series forecasting and anomaly detection recommends Nixtla and sktime (Löning et al. 2019) as good implementations. Nixtla is big! Rob recommends starting from
- statsforecast: Automatic ARIMA and ETS forecasting, after the R
forecastpackage (Rob J. Hyndman and Khandakar 2008) - hierarchicalforecast: Hierarchical forecasting (Olivares et al. 2022)
- tsfeatures: Time series features
Darts (source) (Herzen et al. 2022) aims to be a scikit-learn for time series forecasting. It includes many other algorithms such as prophet, Challu et al. (2022), Oreshkin et al. (2020a), and Salinas, Flunkert, and Gasthaus (2019).
5.3 Foundation models
OK, transformers are everywhere now, and they are predictive time series models—albeit of data like words. Nonetheless, can we hope they might predict more quotidian stuff? Yes. Recent examples include Chronos-2 (Amazon) (Ansari et al. 2025), TimesFM (Google) (Das et al. 2024), Toto (Datadog) (Khwaja et al. 2026), and Timestral (Mistral; no paper yet). These models mostly emit quantiles rather than point forecasts, meaning we can calibrate them like proper Bayesian models.
AutoGluon-TimeSeries (Shchur et al. 2023), Amazon’s AutoML forecasting library, fits and ensembles statsforecast ETS/ARIMA, LightGBM, GluonTS deep models, and Chronos-2 in one call. This makes it an easy way to backtest a foundation model against classical ones on our own data.
See transformers for time series for more.
5.4 prophet
prophet (R/Python/Stan) (S. J. Taylor and Letham 2017):
is a procedure for forecasting time series data. It is based on an additive model where non-linear trends are fit with yearly and weekly seasonality, plus holidays. It works best with daily periodicity data with at least one year of historical data. Prophet is robust to missing data, shifts in the trend, and large outliers.
Commentary:
Is Facebook’s “Prophet” the Time-Series Messiah, or Just a Very Naughty Boy? via Sean J. Taylor on Twitter
This post rips Prophet (a forecasting package I helped create) to shreds and I agree with most of it🥲. I always suspected the positive feedback was mostly from folks who’d had good results—conveniently the author has condensed many bad ones into one place.
5.5 Silverkite
Hosseini et al. (2021)
The Greykite library provides flexible, intuitive and fast forecasts through its flagship algorithm, Silverkite.
Silverkite algorithm works well on most time series, and is especially adept for those with changepoints in trend or seasonality, event/holiday effects, and temporal dependencies. Its forecasts are interpretable and therefore useful for trusted decision-making and insights.
The Greykite library provides a framework that makes it easy to develop a good forecast model, with exploratory data analysis, outlier/anomaly preprocessing, feature extraction and engineering, grid search, evaluation, benchmarking, and plotting. Other open source algorithms can be supported through Greykite’s interface to take advantage of this framework, as listed below.
5.6 Causal impact
🚧TODO🚧 find out how Causal impact works. (Based on Brodersen et al. (2015).)
5.7 asap
Automatic Smoothing for Attention Prioritization in Time Series
ASAP automatically smooths time series plots to remove short-term noise while retaining large-scale deviations.
6 Leaderboards and benchmarks
Popular leaderboards include
- GIFT-Eval (Salesforce): 23 datasets, about 144,000 series, seven domains (Aksu et al. 2024). It publishes a companion pretraining corpus, GiftEvalPretrain.
- fev-bench (AutoGluon/Amazon): 100 tasks across seven domains, 46 of them with covariates (Shchur et al. 2025).
- TIME (Qiao et al. 2026).
I think that is where the action is these days. Standard cautions apply regarding benchmarks in ML.
fev-bench runs as baselines: AutoETS, AutoARIMA and AutoTheta from statsforecast, each fitted per series, plus SCUM (Simple Combination of Univariate Models) (Petropoulos and Svetunkov 2020). These results show every pretrained model is more accurate and faster than the best of the classical models (Shchur et al. 2025) (i.e. SCUM).
Caveats in the classical models’ favour: their context was truncated to 1,000 steps, and 6 hours’ execution time—a limit some of them exceeded.
The older benchmarks below are of only historical interest now. See also Rob J. Hyndman (2020), a short history of forecasting competitions.
6.1 Makridakis competitions
M4 is described in Makridakis, Spiliotis, and Assimakopoulos (2020):
The M4 dataset is a collection of 100,000 time series used for the fourth edition of the Makridakis forecasting Competition. The M4 dataset consists of time series of yearly, quarterly, monthly and other (weekly, daily and hourly) data, which are divided into training and test sets. The minimum numbers of observations in the training test are 13 for yearly, 16 for quarterly, 42 for monthly, 80 for weekly, 93 for daily and 700 for hourly series. The participants were asked to produce the following numbers of forecasts beyond the available data that they had been given: six for yearly, eight for quarterly, 18 for monthly series, 13 for weekly series and 14 and 48 forecasts respectively for the daily and hourly ones.
- M4 Dataset | Papers With Code
- Mcompetitions/M4-methods: Data, Benchmarks, and methods submitted to the M4 forecasting competition
- M4 Forecasting Competition | Rob J Hyndman
- M4 – MOFC
Then came M5, covering hierarchical Walmart retail sales (Makridakis, Spiliotis, and Assimakopoulos 2022), and M6, focusing on financial forecasting and investment decisions (Makridakis et al. 2023).
6.2 Micropredictions.org
micropredictions is a quixotic project my colleagues have shared with me. Included here as a spur. The micropredictions FAQ says:
What’s microprediction you say?
The act of making thousands of predictions of the same type over and over again. Microprediction can
- Clean and enrich live data
- Alert you to outliers and anomalies
- Provide you short term forecasts
- Identify patterns in model residuals
Moreover it can be combined with patterns from Control Theory and Reinforcement Learning to
- Engineer low cost but tailored intelligent applications
Often enough AI is microprediction, albeit bundled with other mathematical or application logic.
- You publish a live data value.
- The sequence of these values gets predicted by a swarm of algorithms.
- Anyone can write a crawler that tries to predict many different streams.
Microprediction APIs make it easy to:
- Separate the act of microprediction from other application logic.
- Invite contribution from other people and machines
- Benefit from other data you may never have considered.
… Let’s say your store is predicting sales and I’m optimising an HVAC system across the street. Your feature space and mine probably have a lot in common.
I am unclear how the datastreams as set up incorporate domain knowledge and private side information, which seem the hallmarks of natural intelligence and, e.g., science. Perhaps they feel domain knowledge is a bug standing in the way of truly general artificial intelligence? If I had free time, I might try to get a better grip on what they are doing, whoever they are.
Alternatively, they are coming at this from a chartist quant perspective, and data are best considered as sort-of-anonymous streams of numbers, the better to attract disinterested competition.
NGL, this always seemed bonkers to me, and now it is defunct.
7 Incoming
Darts:
ARiMA is not Sufficient, an interview on S. Wang, Li, and Lim (2021)

