TAIL-MIL: Time-Aware and Instance-Learnable Multiple Instance Learning for Multivariate Time Series Anomaly Detection

Jaeseok Jang, Hyuk-Yoon KwonAAAI 2025 · OpenReview

Abstract

This study addresses the challenge of detecting anomalies in multivariate time series data. Considering a bag (e.g., multi-sensor data) consisting of two-dimensional spaces of time points and multivariate instances (e.g., individual sensors), we aim to detect anomalies at both the bag and instance level with a unified model. To circumvent the practical difficulties of labeling at the instance level in such spaces, we adopt a multiple instance learning (MIL)-based approach, which enables learning at both the bag- and instance- levels using only the bag-level labels. In this study, we introduce time-aware and instance-learnable MIL (simply, TAIL-MIL). We propose two specialized attention mechanisms designed to effectively capture the relationships between different types of instances. We innovatively integrate these attention mechanisms with conjunctive pooling applied to the two-dimensional structure at different levels (i.e., bag- and instance-level), enabling TAIL-MIL to effectively pinpoint both the timing and causative multivariate factors of anomalies. We provide theoretical evidence demonstrating TAIL-MIL's efficacy in detecting instances with two-dimensional structures. Furthermore, we empirically validate the superior performance of TAIL-MIL over the state-of-the-art MIL methods and multivariate time-series anomaly detection methods.

Introduction: The "Needle in a Haystack" Problem

Detecting anomalies in multivariate time series—such as data from a factory with hundreds of sensors—is a significant challenge. Ideally, a detection system should answer three questions: Is the overall sequence abnormal? At what time did the anomaly occur? And which specific sensor (variate) caused it?

The primary bottleneck is labeling. While experts can often identify a faulty 10-minute window (a "bag"), labeling every individual sensor at every millisecond is practically impossible. Existing Multiple Instance Learning (MIL) methods, which learn from bag labels to predict instance labels, often struggle with the two-dimensional nature (Time $\times$ Variates) of this data. They frequently treat time points as independent or fail to account for temporal dependencies within a single sensor's stream, resulting in "black-box" models that lack interpretability.

Methodology: TAIL-MIL Architecture

TAIL-MIL addresses these issues by treating multivariate time series as a nested hierarchy:

  1. TAB (Time-Aware Bag): The entire multivariate sequence.
  2. TPIM (Time-Point Instance with Multivariate): A single slice of time containing all sensor readings.
  3. VI-TPIM (Variate Instance in TPIM): A single sensor reading at a specific time.
The TAIL-MIL framework utilizes specialized attention and pooling to process nested multivariate time-series data.

To navigate this structure, the framework employs two novel attention mechanisms:

  • T-Attention (Time-Attention): Captures correlations between different time points across the entire bag.
  • V-Attention (Variate-Attention): Focuses on individual sensors, utilizing "Recurrent Features" to incorporate temporal dependencies within a single sensor's stream.

These components are integrated using Conjunctive Pooling. Following a logical "OR" gate logic, if any single sensor at any time point is flagged as anomalous, the entire time point is flagged, and subsequently, the entire bag. This structure ensures that high-level predictions are directly tied to specific, identifiable low-level anomalies.

Key Insights: Breaking the 1D Barrier

Most MIL models are "1D," collapsing multivariate data into a single vector. TAIL-MIL maintains the 2D structure (Time $\times$ Variate), preventing "feature contamination" where noise in one sensor might mask an anomaly in another. Furthermore, the authors provide theoretical evidence showing that the contributions of individual sub-instances in TAIL-MIL are proportional to their Shapley values, providing a mathematically sound basis for the model's explainability.

Experimental Results

The model was evaluated on the State of Health (SOH) battery dataset and the 5-ECK-2022 energy consumption dataset. TAIL-MIL demonstrated superior performance across all levels of the hierarchy, even without sensor-level labels during training.

Performance comparison of TAIL-MIL against other MIL methods at the bag (TAB) and time-point (TPIM) levels.

Key performance highlights include:

  • Bag-Level (TAB): On the SOH dataset, TAIL-MIL achieved an AUROC of 0.9574, nearly matching supervised models.
  • Time-Point Level (TPIM): It maintained a high AUROC of 0.9539, significantly outperforming 1D MIL models like TimeMIL (~0.58), which struggle with multivariate complexity.
  • Variate-Level (VI-TPIM): TAIL-MIL reached an AUROC of 0.9290, a substantial leap over Nested MIL (0.7272), confirming its ability to identify causative factors.

Conclusion

TAIL-MIL offers a powerful, interpretable solution for multivariate time series anomaly detection. By leveraging a 2D nested structure and specialized attention mechanisms, it provides fine-grained insights using only coarse bag-level labels. Future work may explore its application in other 2D domains like video analysis or medical imaging, as well as optimizing computational efficiency for extremely high-dimensional datasets.

Summary updated

Abstract

This study addresses the challenge of detecting anomalies in multivariate time series data. Considering a bag (e.g., multi-sensor data) consisting of two-dimensional spaces of time points and multivariate instances (e.g., individual sensors), we aim to detect anomalies at both the bag and instance level with a unified model. To circumvent the practical difficulties of labeling at the instance level in such spaces, we adopt a multiple instance learning (MIL)-based approach, which enables learning at both the bag- and instance- levels using only the bag-level labels. In this study, we introduce time-aware and instance-learnable MIL (simply, TAIL-MIL). We propose two specialized attention mechanisms designed to effectively capture the relationships between different types of instances. We innovatively integrate these attention mechanisms with conjunctive pooling applied to the two-dimensional structure at different levels (i.e., bag- and instance-level), enabling TAIL-MIL to effectively pinpoint both the timing and causative multivariate factors of anomalies. We provide theoretical evidence demonstrating TAIL-MIL's efficacy in detecting instances with two-dimensional structures. Furthermore, we empirically validate the superior performance of TAIL-MIL over the state-of-the-art MIL methods and multivariate time-series anomaly detection methods.
BibTeX
@inproceedings{jang2025tail,
  author = {Jaeseok Jang and Hyuk-Yoon Kwon},
  title = {{TAIL-MIL: Time-Aware and Instance-Learnable Multiple Instance Learning for Multivariate Time Series Anomaly Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=Efnx4U5S0W},
}
Figure