Abstract
Unsupervised model recommendation for anomaly detection is a recent discipline for which there is no existing work that focuses on multivariate time series data. This paper studies that problem under real-world restrictions, most notably: (i) a limited time to issue a recommendation, which renders existing methods based around the testing of a large pool of models unusable; (ii) the need for generalization to previously unseen data sources, which is seldom factored in the experimental evaluation. We turn to meta-learning and propose Orthus (also referred to as Hydra), the first meta-recommender for anomaly detection in literature that we especially analyze in the context of multivariate times series. We conduct our experiments using 94 public datasets from 4 different data sources. Our ablation study testifies that our meta-recommender achieves a higher performance than the current state of the art, including in difficult scenarios in which data similarity is minimal: our proposal is able to recommend a model in the top 10% (13%) of the algorithmic pool for known (unseen) sources of data.
Introduction
Finding anomalies in multivariate time series data—such as sudden spikes in CPU usage across a server cluster or sensor malfunctions in industrial plants—is a classic "No Free Lunch" problem. No single algorithm consistently outperforms others across all datasets. In practice, anomaly detection is often unsupervised because ground truth labels are unavailable. This creates a significant challenge: how can one select the optimal model without performance verification?
Traditional model selection involves testing a large pool of algorithms, which is computationally prohibitive for real-time applications. Furthermore, most existing recommenders fail to generalize when encountering "novel" data sources that differ significantly from their training data.
The Orthus Framework
To address these challenges, we introduce Orthus, a "two-headed" meta-recommender designed for speed and robust generalization. The framework is built upon three primary pillars:
- The Novelty Switch: Upon receiving a new dataset, Orthus utilizes UMAP for dimensionality reduction and DBSCAN for clustering to determine if the data resembles previously seen sources.
- Head 1: Clustered Factorization (CFact): For "known" data, Orthus employs a refined recommender that clusters similar algorithmic configurations to provide highly tuned suggestions.
- Head 2: URegression (UReg): For "novel" data, the system switches to a more robust approach using Singular Value Decomposition (SVD). This captures broad performance patterns, preventing overfitting to specific characteristics of the training domain.
Additionally, Orthus utilizes Catch22 meta-features—a set of 22 fast-to-compute time series characteristics summarized into 110 values. This approach is four times faster than previous state-of-the-art methods like MetaOD and proves more effective at capturing general temporal characteristics relevant to model performance.
Experimental Setup and Datasets
Our study utilizes a comprehensive benchmark of 94 datasets from four distinct public sources:
- BGP: Network telemetry data with injected anomalies.
- SMD (Server Machine Dataset): Resource metrics from internet companies.
- Water: Sensor data from water-treatment facilities.
- Benchmark: A collection of synthetic and realistic outlier detection datasets.
As shown in the UMAP projection, these sources exhibit very little overlap. This separation is critical for testing the framework's ability to generalize to unseen domains, simulating real-world "cold-start" scenarios.
Key Results
Orthus demonstrates superior performance across both familiar and novel data environments:
- High Accuracy: In scenarios with known data sources, Orthus selects models within the top 10% of all possible configurations. For completely unseen sources, it still identifies models in the top 13%.
- Robust Generalization: While competing methods like MetaOD see performance drops to roughly 60% (average percentile) on novel data, Orthus maintains a high performance of approximately 87%.
- Operational Speed: The recommendation phase is completed in roughly 25 milliseconds, making it suitable for production environments requiring near-instantaneous model deployment.
Conclusion
Orthus represents a significant advancement in unsupervised model recommendation for multivariate time series. By distinguishing between familiar and novel data and utilizing efficient meta-features, it provides high-quality recommendations at speeds required for industrial applications. Future work will explore the massive design space of meta-features and alternative optimization targets beyond PRAUC to further refine recommendation accuracy.
Summary updated