Abstract
Detecting and classifying abnormal system states is critical for condition monitoring, but supervised methods often fall short due to the rarity of anomalies and the lack of labeled data. Therefore, clustering is often used to group similar abnormal behavior. However, evaluating cluster quality without ground truth is challenging, as existing measures such as the Silhouette Score (SSC) only evaluate the cohesion and separation of clusters and ignore possible prior knowledge about the data. To address this challenge, we introduce the Synchronized Anomaly Agreement Index (SAAI), which exploits the synchronicity of anomalies across multivariate time series to assess cluster quality. We demonstrate the effectiveness of SAAI by showing that maximizing SAAI improves accuracy on the task of finding the true number of anomaly classes K in correlated time series by 0.23 compared to SSC and by 0.32 compared to X-Means. We also show that clusters obtained by maximizing SAAI are easier to interpret compared to SSC.
Introduction
In complex systems—such as spacecraft or high-tech greenhouses—monitoring for anomalies is vital for maintaining operational integrity. However, because anomalies are rare, obtaining sufficient labeled data to train supervised classifiers is often impossible. Instead, engineers rely on unsupervised clustering to group similar abnormal behaviors. A critical challenge in this process is determining the ideal number of clusters ($K$).
Standard unsupervised metrics like the Silhouette Score (SSC) or X-Means often fail when applied to raw time series data. These tools focus purely on mathematical distances in feature space but ignore the "prior knowledge" inherent in the temporal structure of the data. Specifically, they overlook the temporal relationship between different sensor channels: if multiple sensors react at the exact same time, they are likely responding to the same physical event. Current metrics lack a mechanism to account for this "synchronized agreement."
Methodology
To address these limitations, we propose the Synchronized Anomaly Agreement Index (SAAI). SAAI is an internal cluster quality measure that does not require ground truth labels. It is built upon two main components:
- Synchronicity Ratio: This calculates how often anomalies that occur simultaneously (temporally aligned) across different dimensions are assigned to the same cluster.
- Regularization: A penalty term is included to discourage the formation of "pseudo-clusters" (clusters containing only a single anomaly) and excessively small cluster sizes.
The SAAI formula uses a parameter $\lambda$ to balance these terms. To calculate it efficiently, we developed a sweep-line algorithm that identifies overlapping anomalous subsequences across different dimensions of a multivariate time series.
Experimental Setup
The study utilizes two primary data sources to validate the metric:
- Synthetic ICS Dataset: A simulated dataset mimicking the Illumination Control System (ICS) of the EDEN ISS (an experimental greenhouse in Antarctica). It includes periodic signals with injected red noise and six specific types of anomalies (e.g., "Long Day Peak," "Night Drop").
- Real EDEN ISS 2020 Telemetry: Actual temperature measurements from 38 sensors in the Antarctic greenhouse. This provides a real-world environment where the true number of anomaly types was previously estimated by experts to be approximately ten.
Results and Discussion
Our experiments reveal that temporal alignment is a powerful substitute for labels in correlated systems. Simultaneous shocks are rarely coincidental; they represent different "views" of the same underlying problem.
Key results include:
- Accuracy Boost: On the task of finding the true number of anomaly classes ($K$), maximizing SAAI improved accuracy by 0.23 compared to the Silhouette Score and by 0.32 compared to X-Means.
- High Correlation with External Metrics: SAAI results were highly correlated with the Adjusted Rand Index (ARI) and Fowlkes-Mallows Index (FMI)—metrics that require ground truth labels—proving that SAAI is a reliable unsupervised stand-in.
- Real-World Interpretability: When applied to the real EDEN ISS data, SAAI identified 11 clusters, closely matching the 10 types identified by experts. In contrast, the Silhouette Score suggested only 3 clusters, which were visually messy and difficult to interpret.
- Correlation Threshold: Through "Lagged Variable" experiments, we found that SAAI is most effective when the pairwise correlation between signals is $\rho \geq 0.5$.
Conclusion and Future Work
While SAAI significantly improves the identification of anomaly classes in correlated multivariate time series, it has certain limitations. It currently focuses on univariate anomalies within multivariate series and does not yet account for anomalies that only manifest through the interaction of multiple variables. Additionally, it ignores "lone wolf" anomalies that occur in only one channel or are not temporally aligned. Future research will aim to extend SAAI to handle truly multivariate anomalies and incorporate unaligned events to expand its applicability to less correlated datasets.
Summary updated