TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoders

Mingyue Cheng, Qi Liu, Zhiding Liu, Hao Zhang, Rujiao Zhang, Enhong Chen

Abstract

TimeMAE is a novel self-supervised framework designed to learn robust time series representations by treating sub-series "patches" as semantic units. By employing a decoupled architecture that excludes artificial mask tokens from the primary encoder and utilizing a dual-target reconstruction strategy—incorporating both discrete codewords and continuous features—TimeMAE establishes new benchmarks for time series classification. It demonstrates particular strength in data-scarce environments and transfer learning scenarios.

Introduction

Deep learning models, especially Transformers, typically require vast amounts of labeled data to perform effectively. In domains such as medical monitoring or industrial sensor analysis, labeled data is often prohibitively expensive or rare. Self-supervised learning (SSL) offers a solution by pre-training models on unlabeled data. However, existing time series SSL methods face three significant hurdles:

  1. Low Semantic Density: Masking individual time steps often allows models to "cheat" by interpolating between neighbors, failing to capture high-level temporal concepts.
  2. The "Mask Token" Mismatch: Standard masked autoencoders (MAEs) insert [MASK] tokens during pre-training that never appear during fine-tuning, creating a distribution shift that degrades performance.
  3. Point-wise Focus: Discriminative patterns in time series are usually "shapelets" (sub-series) rather than isolated data points.

TimeMAE addresses these challenges by rethinking the MAE philosophy specifically for temporal data.

Methodology

TimeMAE introduces three core innovations to transform self-supervised learning for time series:

Window Slicing (Patching)

Instead of masking individual points, TimeMAE segments the series into non-overlapping sub-series or "patches." This forces the model to reconstruct meaningful temporal chunks, increasing the semantic depth of the task while reducing the computational complexity of the Transformer's self-attention mechanism.

Decoupled Masked Autoencoder

To eliminate the mask token mismatch, TimeMAE utilizes a decoupled architecture. The Online Encoder only processes visible (unmasked) patches. A separate Decoupled Decoder uses cross-attention, treating the encoder's output as "keys/values" and using mask tokens only as "queries" to predict missing segments. This ensures the encoder never encounters a mask token during pre-training, aligning its behavior perfectly with downstream fine-tuning tasks.

The TimeMAE architecture featuring decoupled encoders and dual learning objectives.

Dual Learning Objectives

The framework employs two complementary loss functions:

  • Masked Codeword Classification (MCC): The model learns a "tokenizer"—a codebook of prototypical temporal patterns—and attempts to classify which codeword best represents a masked patch.
  • Masked Representation Regression (MRR): The model matches the continuous feature representation of the masked patch, guided by a Target Encoder updated via a slow-moving average (momentum update) of the online encoder.

Experimental Setup

The authors evaluated TimeMAE across five diverse multivariate time series datasets: Human Activity Recognition (HAR), Phoneme Spectra (PS), ArabicDigits (AD), Uwave (gesture recognition), and Epilepsy (EEG monitoring). Performance was measured in two modes: FineLast (linear probing with a frozen encoder) and FineAll (full fine-tuning).

Results and Discussion

Superior Accuracy and Efficiency

TimeMAE consistently outperformed competitive baselines such as TST, TS2Vec, and SimMTM. On the HAR dataset, it achieved an accuracy of 95.11%, a significant improvement over the 89.73% achieved by a supervised-from-scratch baseline.

Label Efficiency

One of TimeMAE's most striking features is its performance in low-data regimes. As shown in the experimental results, when only 3% to 10% of labels are available, TimeMAE maintains high accuracy while competing models experience sharp declines. This robustness makes it ideal for real-world applications where labeling is a bottleneck.

Performance comparison of TimeMAE against baselines across varying percentages of labeled data.

Transfer Learning and Semantic Elevation

By moving from points to patches, the model effectively learns "shapelets"—the fundamental building blocks of time series. Furthermore, keeping the encoder "pure" of mask tokens significantly enhances transferability. When pre-trained on HAR and tested on other datasets, TimeMAE demonstrated remarkable zero-shot capabilities, outperforming other SSL methods.

Conclusion

TimeMAE redefines the application of Masked Autoencoders to time series by emphasizing semantic patches and architectural decoupling. While the model involves some computational overhead due to the dual-encoder setup and requires careful hyperparameter tuning, its superior performance in data-scarce and transfer learning scenarios marks a significant step toward universal foundation models for temporal data. Future work may focus on scaling these models to even larger datasets and longer pre-training durations.

Summary updated

BibTeX
@inproceedings{cheng2026timemae,
  author = {Mingyue Cheng and Qi Liu and Zhiding Liu and Hao Zhang and Rujiao Zhang and Enhong Chen},
  title = {{TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoders}},
  booktitle = {WSDM},
  year = {2026},
  doi = {10.1145/3773966.3778007},
}
Figure