TS-OOD: Evaluating Time-Series Out-of-Distribution Detection and Prospective Directions for Progress

Onat Güngör, Amanda Sofie Rios, Nilesh A. Ahuja, Tajana RosingarXiv 2025 · DOI · arXiv

Abstract

Out-of-distribution (OOD) detection is critical for reliable AI, yet it remains under-explored in the time-series domain. This paper introduces TS-OOD, a rigorous evaluation pipeline designed to benchmark OOD detection methods specifically for time-series data. Our findings reveal that many state-of-the-art methods from computer vision fail to translate effectively, while approaches based on Deep Feature Modeling (DFM) emerge as robust and efficient solutions for detecting semantic shifts in diverse time-series applications.

Introduction: The Challenge of Time-Series OOD

Machine learning models often exhibit overconfidence, providing "confidently wrong" predictions when encountering out-of-distribution (OOD) data. In high-stakes time-series applications—such as medical diagnostics or industrial monitoring—this behavior can lead to catastrophic failures. Unlike computer vision or natural language processing, time-series data lacks massive foundational models and presents unique challenges due to its diverse modalities and complex temporal dependencies.

Existing research often relies on "easy" OOD benchmarks, such as using entirely different datasets as OOD samples. This study addresses the more realistic and difficult problem of Semantic Shift: identifying novel, unseen classes within the same data modality.

Methodology: The TS-OOD Pipeline

To systematically evaluate OOD detection, the researchers developed the TS-OOD pipeline, which examines the impact of various deep learning components on detection performance.

The TS-OOD evaluation pipeline, encompassing time-series augmentation, ID training with various backbones and losses, and OOD detection using multiple methods.

The pipeline evaluates four key dimensions:

  1. Backbones: ResNet (1D-CNN), TST (Transformer), and LSTM (Recurrent).
  2. Data Augmentations: Seven techniques, including Jittering, Permutation, and Magnitude Warping.
  3. Loss Functions: Standard Cross-Entropy (CE) versus Multi-Positive Contrastive (MPC) loss.
  4. OOD Detectors: Eight methods ranging from simple softmax scores (MSP) to advanced feature-space modeling like Deep Feature Modeling (DFM) and Mahalanobis Distance (MDS).

The study utilized 13 multivariate time-series datasets from the UCR/UEA repository, covering domains like traffic flow and medical signals. Each dataset was split 50/50 by class to create a challenging semantic shift scenario.

Key Insights: What Drives Performance?

  • Failure of Vision-Centric Methods: Popular OOD methods like ReACT and DICE, which rely on activation patterns common in vision models, performed poorly on time-series architectures.
  • Superiority of Feature Modeling: Methods that model the internal feature distribution—particularly DFM-PCA—consistently outperformed classification-based scores.
  • Benefits of Contrastive Learning: Training with MPC loss improved AUROC by an average of 4.3% compared to CE loss, as it helps the model learn more distinct clusters for known classes.
  • Impact of Augmentation: Magnitude Warping was identified as the most effective augmentation for enhancing OOD sensitivity, though its impact varied across different data modalities.

Results and Analysis

The evaluation highlights significant differences in how various backbones and detectors handle OOD data.

Performance comparison (AUROC and AUPR) of various OOD detection methods across ResNet, TST, and LSTM backbones.

Key findings include:

  • Top Performers: DFM-based methods (DFM-PCA and DFM-SVM) proved most robust across all backbones.
  • Backbone Sensitivity: ResNet and TST architectures are significantly better suited for OOD detection than LSTMs, which struggled to provide reliable uncertainty measurements.
  • Efficiency: DFM methods are highly efficient, with an inference overhead of approximately 0.164 ms per sample, making them ideal for real-time edge applications.
  • Accuracy Correlation: A strong correlation exists between a model's accuracy on in-distribution data and its OOD detection capability when using feature-based methods.

Conclusion and Future Directions

While DFM provides a strong baseline for time-series OOD detection, there is significant room for growth. Future research should explore multi-layer feature extraction to capture anomalies at various levels of abstraction. Furthermore, as the field moves toward large-scale pre-trained "foundation" models for time-series, it remains to be seen whether these models will inherently possess better OOD detection capabilities or if specialized post-hoc methods like those identified in TS-OOD will remain essential.

Summary updated

Abstract

Detecting out-of-distribution (OOD) data is a fundamental challenge in the deployment of machine learning models. From a security standpoint, this is particularly important because OOD test data can result in misleadingly confident yet erroneous predictions, which undermine the reliability of the deployed model. Although numerous models for OOD detection have been developed in computer vision and language, their adaptability to the time-series data domain remains limited and under-explored. Yet, time-series data is ubiquitous across manufacturing and security applications for which OOD is essential. This paper seeks to address this research gap by conducting a comprehensive analysis of modality-agnostic OOD detection algorithms. We evaluate over several multivariate time-series datasets, deep learning architectures, time-series specific data augmentations, and loss functions. Our results demonstrate that: 1) the majority of state-of-the-art OOD methods exhibit limited performance on time-series data, and 2) OOD methods based on deep feature modeling may offer greater advantages for time-series OOD detection, highlighting a promising direction for future time-series OOD detection algorithm development.
BibTeX
@misc{gungor2025ts,
  author = {Onat Güngör and Amanda Sofie Rios and Nilesh A. Ahuja and Tajana Rosing},
  title = {{TS-OOD: Evaluating Time-Series Out-of-Distribution Detection and Prospective Directions for Progress}},
  year = {2025},
  eprint = {2502.15901},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.15901},
}
Figure