TS2Vec: Towards Universal Representation of Time Series

Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, Bixiong Xu

Abstract

This paper presents TS2Vec, a universal framework for learning representations of time series at an arbitrary semantic level. Unlike existing methods, TS2Vec performs contrastive learning in a hierarchical way over augmented context views, which enables a robust contextual representation for each timestamp. Furthermore, to obtain the representation of an arbitrary sub-sequence in the time series, we can apply a simple aggregation over the representations of corresponding timestamps. We conduct extensive experiments on time series classification tasks to evaluate the quality of time series representations. As a result, TS2Vec achieves significant improvement over existing SOTAs of unsupervised time series representation on 125 UCR datasets and 29 UEA datasets. The learned timestamp-level representations also achieve superior results in time series forecasting and anomaly detection tasks. A linear regression trained on top of the learned representations outperforms previous SOTAs of time series forecasting. Furthermore, we present a simple way to apply the learned representations for unsupervised anomaly detection, which establishes SOTA results in the literature.

Introduction

Time series analysis is a cornerstone of industries ranging from finance to climate modeling. However, learning "universal" representations—features that work well for any downstream task—remains difficult. Existing unsupervised methods often suffer from three major gaps: granularity mismatch (focusing only on instance-level summaries), a lack of multi-scale information, and flawed inductive biases borrowed from computer vision that may not apply to temporal data.

TS2Vec addresses these challenges by learning contextual representations without relying on heavy-handed data transformations. Its architecture consists of an input projection layer, a timestamp masking module, and a dilated CNN backbone, designed to generate robust embeddings at any semantic level—from individual timestamps to entire sequences.

Methodology

The core innovation of TS2Vec lies in its approach to consistency and hierarchical learning.

Contextual Consistency

Instead of comparing two different sub-series, TS2Vec generates two "augmented context views" of the same input via random cropping and timestamp masking. The model is trained to ensure that the representation of the same timestamp is consistent across these different contexts. This avoids the pitfalls of transformation invariance, where a cropped sub-sequence might have a completely different distribution than the original.

Comparison of contextual consistency with other consistency-based learning objectives.

Hierarchical Contrasting

The framework applies a contrastive loss at multiple temporal resolutions. By using recursive max-pooling along the time axis, the model learns to discriminate between positive and negative samples at both fine-grained and coarse-grained levels. The objective function combines temporal contrast (comparing the same timestamp in two views) and instance-wise contrast (comparing the same point across different instances in a batch).

Overview of the TS2Vec architecture and hierarchical contrasting framework.

Latent Space Masking

Unlike NLP models that mask input tokens, TS2Vec masks the latent vectors after the projection layer. This is crucial for continuous time series data, as it prevents the model from simply learning to ignore a "zero" or special token value, forcing it to infer missing values from the surrounding context.

Experimental Results

TS2Vec was evaluated on an exhaustive suite of benchmarks, including 128 UCR univariate datasets, 30 UEA multivariate datasets, and various forecasting and anomaly detection archives.

  • Classification: TS2Vec improved average accuracy by 2.4% on UCR and 3.0% on UEA compared to previous unsupervised leaders.
  • Forecasting: A simple linear regression trained on top of frozen TS2Vec representations achieved a 32.6% reduction in Mean Squared Error (MSE) on univariate forecasting compared to previous SOTA models like Informer.
  • Anomaly Detection: On the Yahoo dataset, TS2Vec improved the F1 score by 18.2% over the best existing unsupervised methods.
  • Efficiency: TS2Vec is significantly faster to train than many transformer-based competitors. For example, on the ETTm1 dataset, its training time was orders of magnitude lower than models like TNC.

Conclusion

TS2Vec provides a robust, universal framework for time series representation. By leveraging hierarchical contrastive learning and contextual consistency, it bridges the gap between instance-level and timestamp-level analysis. The model's ability to outperform complex end-to-end architectures with simple linear heads demonstrates the high quality and universality of the learned representations.

Summary updated

Abstract

This paper presents TS2Vec, a universal framework for learning representations of time series in an arbitrary semantic level. Unlike existing methods, TS2Vec performs contrastive learning in a hierarchical way over augmented context views, which enables a robust contextual representation for each timestamp. Furthermore, to obtain the representation of an arbitrary sub-sequence in the time series, we can apply a simple aggregation over the representations of corresponding timestamps. We conduct extensive experiments on time series classification tasks to evaluate the quality of time series representations. As a result, TS2Vec achieves significant improvement over existing SOTAs of unsupervised time series representation on 125 UCR datasets and 29 UEA datasets. The learned timestamp-level representations also achieve superior results in time series forecasting and anomaly detection tasks. A linear regression trained on top of the learned representations outperforms previous SOTAs of time series forecasting. Furthermore, we present a simple way to apply the learned representations for unsupervised anomaly detection, which establishes SOTA results in the literature. The source code is publicly available at https://github.com/yuezhihan/ts2vec.
BibTeX
@inproceedings{yue2022ts,
  author = {Zhihan Yue and Yujing Wang and Juanyong Duan and Tianmeng Yang and Congrui Huang and Yunhai Tong and Bixiong Xu},
  title = {{TS2Vec: Towards Universal Representation of Time Series}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i8.20881},
}
Figure