TimesFM: Applying the NLP Foundation-Model Paradigm to Time-Series Forecasting
Over the past 50 years, time-series forecasting has evolved from ARIMA to XGBoost, from XGBoost to LSTM, and from LSTM to Transformer. Each step got more complex, yet none solved a fundamental problem: you still had to train a new model from scratch for every scenario. Sales forecasting, power load forecasting, server traffic forecasting—each requires labeled data, feature engineering, and hyperparameter tuning. A new business line could take data scientists weeks to produce a mediocre forecast.
TimesFM's contribution is bringing the NLP "pretraining + zero-shot generalization" paradigm to time series. No labeled data, no feature engineering—give it historical data, and it predicts the future directly.
Decoder-only: Why Time Series Doesn't Need an Encoder
TimesFM's most counterintuitive design choice is its decoder-only architecture.
In NLP, decoder-only models (the GPT family) succeed because language generation is essentially "predict the next token given prior context." But isn't time-series forecasting the same—"predict the next value given history"?
Yes, with one key difference: language is discrete (tokens are discrete), while time series are continuous. You can't feed continuous values directly as tokens.
TimesFM's solution is patch-based quantization: the continuous series is split into patches of length 32, each quantized into a token. A 1024-point series thus becomes 32 tokens that can be fed straight into a decoder-only model.
This lets TimesFM reuse the LLM training recipe: large-scale unsupervised pretraining + autoregressive generation. No labeled data needed—just lots of time-series data.
The "Small Model" Philosophy: 200M Parameters
TimesFM 2.5 has 200M parameters. For comparison, GPT-4 has 1.7 trillion.
Why is 200M enough? Because time series have far lower information density than natural language. 1024 tokens of text can contain a complete story with plot, emotion, and logic. 1024 points of a time series might just be a server's CPU utilization curve.
The TimesFM team found that 200M parameters already capture the statistical patterns of time series. Larger models brought no significant performance gains, only higher inference costs.
This design decision means TimesFM runs on ordinary GPUs, even Apple Silicon. You don't need an A100 cluster for forecasting—a 200M model runs on a laptop.
16K Context: From Short-Term to Long-Term Trends
Another TimesFM 2.5 upgrade: context length increased from 2048 to 16K points.
This isn't just a numbers game. A 2048-point context (~32 patches) only covers a few days of minute-level data—enough for short-term forecasting. 16K points covers months of data, capturing periodic patterns: weekly cycles, monthly cycles, even seasonality.
For real business scenarios this matters greatly. E-commerce needs next-month sales forecasts (monthly cycles); utilities need summer peak-load forecasts (seasonal cycles)—both demand longer context windows.
Zero-Shot Forecasting: Train on Scenario A, Infer on Scenario B
TimesFM's core promise is zero-shot generalization: after pretraining on large amounts of public time-series data, it forecasts new scenarios without fine-tuning.
According to the paper, TimesFM's zero-shot performance approaches fully supervised models. That means you can take a pretrained TimesFM and apply it directly to your business—no labeled data, no new training.
Google has integrated TimesFM into three products:
- BigQuery ML: enterprise SQL queries, forecasting directly in the data warehouse
- Google Sheets: time-series forecasting inside spreadsheets
- Vertex Model Garden: Dockerized endpoints supporting agentic invocation
- Sales forecasting: promotion calendars, weather data
- Power load forecasting: temperature, humidity
- Server traffic forecasting: deployment events, marketing campaigns
- GitHub: https://github.com/google-research/timesfm
- Paper: https://arxiv.org/abs/2310.10688
- HuggingFace: https://huggingface.co/collections/google/timesfm-release
- Google Research Blog: https://research.google/blog/
The short path from research to product suggests Google has confidence in TimesFM's practicality.
Covariate Support: Not Just Historical Values
TimesFM 2.5 brings back covariate support (via XReg). Beyond the target variable, you can supply external features:
Covariates evolve TimesFM from "pure zero-shot forecasting" to "zero-shot + external-information enhancement." You still need no labeled data for training, but you can provide additional context at inference time.
Paradigm Transfer of Foundation Models
TimesFM's significance goes beyond being "yet another time-series model." It validates a bigger hypothesis: the foundation-model paradigm can transfer from NLP to other domains.
The NLP paradigm—large-scale unsupervised pretraining + few/zero-shot generalization—was validated by GPT/LLaMA. TimesFM applies the same recipe to time series: pretraining on massive time-series data + zero-shot forecasting, with a decoder-only architecture, autoregressive objective, and quantized patch tokens as input.
The success of this transfer implies something: the core of the foundation-model paradigm isn't "language"—it's "sequences." Any data that can be split into token sequences may be modeled this way.
From NLP to time series, the next targets might be protein sequences, musical melodies, or code execution traces. TimesFM isn't an endpoint—it's a milestone in paradigm transfer.
---
Related links: