The Language of Time: a Language Model Perspective on Time Series Foundation Models
Lists
Only you see this
Abstract
Large language models have established a successful paradigm of training foundation models on massive datasets, extending this approach to multiple domains. Time series foundation models extend this paradigm, demonstrating exceptional cross-domain transfer and prediction capabilities in both industrial and academic scenarios. This creates a paradox: while time series from different domains reflect distinct dynamical systems that should limit transferability, empirical results demonstrate remarkable cross-domain performance. To resolve this paradox, this paper investigates the representation learning mechanisms and generalization capabilities of patch-based time series foundation models from both theoretical and experimental perspectives. We demonstrate that these models fundamentally extend language model representations from deterministic vectors to probabilistic distributions, enabling effective cross-domain transfer. Our analysis shows that time series patches can be quantized into discrete vocabularies with statistical properties similar to natural language. This theoretical framework explains how time series models inherit the robust representation and transfer abilities of large language models, accounting for their superior performance in temporal tasks. Our work provides a rigorous theoretical foundation for understanding, evaluating, and improving the safety and reliability of large-scale time series foundation models for time series analysis.
BibTeX
@misc{xie2025language,
author = {Yi Xie and Yun Xiong and Ziyue Li and Hao Niu and Zejian Shi and Sijia Peng and Zhengfu Liu},
title = {{The Language of Time: a Language Model Perspective on Time Series Foundation Models}},
year = {2025},
eprint = {2507.00078},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2507.00078},
}