Time Series Forecasting the NLP Way: TimesFM Brings the Foundation Model Paradigm to a New Domain

Over the past 50 years, progress in time series forecasting can be summarized in one sentence: from ARIMA to XGBoost, from XGBoost to LSTM, from LSTM to Transformer.

Each step became more complex, but none solved a fundamental problem: you still had to train a model from scratch for every new scenario. E-commerce sales forecasting, power load forecasting, server traffic forecasting—each scenario requires labeled data, feature engineering, and hyperparameter tuning. When a new business launches, data scientists spend weeks producing a barely acceptable forecast.

What TimesFM does is bring the NLP paradigm of "pretraining + zero-shot generalization" to time series. No labeled data, no feature engineering—give it historical data, and it predicts the future directly.

Decoder-only: Why Time Series Doesn't Need an Encoder

TimesFM's most counterintuitive design choice is its decoder-only architecture.

In NLP, decoder-only models (the GPT family) succeed because language generation is essentially "predict the next token given preceding context." But isn't time series forecasting also "predict the next value given history"?

Yes—but with one key difference: language is discrete (tokens are discrete), while time series is continuous. You can't feed continuous values directly as tokens.

TimesFM's solution is patch-wise quantization: the continuous time series is cut into chunks of length 32, and each chunk is quantized into a token. A time series of length 1024 thus becomes 32 tokens, which can be fed directly into a decoder-only model.

This design lets TimesFM reuse the LLM training recipe directly: large-scale unsupervised pretraining + autoregressive generation. No labeled data needed—just massive amounts of time series data.

The Philosophy of a "Small" 200M-Parameter Model

TimesFM 2.5 has 200M parameters. For comparison, GPT-4 has 1.7 trillion.

Why is 200M enough? Because the information density of time series is far lower than natural language. A 1024-token text might contain a complete story—plot, emotion, logical relationships. A 1024-point time series might just be a server's CPU utilization curve, with limited information.

The TimesFM team found that 200M parameters are already enough to fit the statistical regularities of time series. Larger models brought no significant performance gains while increasing inference cost.

This design decision means TimesFM runs on ordinary GPUs and even supports Apple Silicon. You don't need an A100 cluster for time series forecasting—a 200M model runs on a laptop.

16K Context: From Short-Term Forecasting to Long-Term Trends

Another TimesFM 2.5 upgrade is the context length increase from 2048 to 16K.

This is not just a numbers game. A context of 2048 points (~32 chunks) only covers a few days of minute-level data—fine for short-term forecasting. 16K points can cover several months, capturing periodic patterns: weekly cycles, monthly cycles, even seasonal cycles.

For real business scenarios, this matters enormously. E-commerce needs next month's sales forecast (monthly cycle); utilities need to predict summer peak demand (seasonal cycle)—both require longer context windows.

Zero-Shot Forecasting: Trained on Scenario A, Inferred on Scenario B

TimesFM's core promise is zero-shot generalization: after pretraining on massive public time series data, it forecasts on new scenarios directly, without fine-tuning.

Per the paper's experiments, TimesFM's zero-shot performance approaches fully supervised models. What does that mean? You can take a pretrained TimesFM and apply it directly to your business scenario—no labeled data, no training a new model.

Google has already integrated TimesFM into three products:

• BigQuery ML: enterprise SQL queries, forecasting directly inside the data warehouse
• Google Sheets: time series forecasting in spreadsheets
• Vertex Model Garden: Dockerized endpoints supporting agentic invocation

The short path from research to product suggests Google is confident in TimesFM's practicality.

Covariate Support: Not Just Looking at Historical Values

TimesFM 2.5 reintroduces covariate support (via XReg). Beyond the target variable itself, you can feed in external features:

• Sales forecasting: add promotion calendars, weather data
• Power load forecasting: add temperature, humidity
• Server traffic forecasting: add deployment events, marketing campaigns

Covariate support evolves TimesFM from "pure zero-shot forecasting" to "zero-shot + external information augmentation." You still don't need labeled data for training, but you can provide additional context at inference time.

Paradigm Transfer of Foundation Models

TimesFM's significance isn't just "yet another time series model." It validates a bigger hypothesis: the foundation model paradigm can transfer from NLP to other domains.

The NLP foundation model paradigm: large-scale unsupervised pretraining + few-shot/zero-shot generalization—a paradigm already validated in NLP by GPT, LLaMA, and similar models.

TimesFM applies the same paradigm to time series: pretraining on massive time series data + zero-shot forecasting. Decoder-only architecture, autoregressive training objective, quantized patch tokens as input.

The success of this transfer implies something: the core of the foundation model paradigm isn't "language"—it's "sequences." Any data that can be segmented into a token sequence may be modeled with this paradigm.

From NLP to time series, the next candidates might be protein sequences, musical melodies, or code execution traces. TimesFM isn't the endpoint—it's a milestone in paradigm transfer.

---

Related links:

• GitHub: https://github.com/google-research/timesfm
• Paper: https://arxiv.org/abs/2310.10688
• HuggingFace: https://huggingface.co/collections/google/timesfm-release
• Google Research Blog: https://research.google/blog/