Introduction
Anomaly detection is a critical component in cybersecurity and industrial monitoring, yet practitioners often face significant challenges in real-world deployments. Most existing systems rely on a single unsupervised algorithm, which can fail when data patterns shift or attackers adapt. Furthermore, while theoretical models often assume immediate feedback, real-world ground-truth labels are frequently delayed by days or weeks. This paper introduces a "superexpert" framework that aggregates multiple unsupervised models and adapts to delayed feedback while maintaining strong theoretical performance guarantees.
Problem Statement
Two primary hurdles limit the effectiveness of current anomaly detection:
- Algorithm Volatility: No single model remains optimal indefinitely. A model effective for network intrusions might fail during baseline drifts.
- The Feedback Gap: Labels are rare and often arrive long after a prediction is made (e.g., after an analyst's weekly review).
Traditional "Prediction with Expert Advice" (PEA) frameworks typically require immediate feedback. This research extends these theories to handle non-stationary environments and significant feedback delays by allowing the ensemble to switch between experts dynamically.
Methodology
The core of the proposed system is the Aggregating Algorithm (AA), which maintains weights for a pool of "experts" (base anomaly detection models). The authors introduce several key innovations to handle real-world constraints:
- AAP-current (Pack Averages): Adapts the AA for "packs" of outcomes that arrive after a delay, rather than updating on single immediate outcomes.
- Fixed-share & Variable-share: These mechanisms allow the ensemble to "switch" its allegiance between experts. Fixed-share redistributes a portion of weights at each step, making it ideal for environments where the best model changes periodically. Variable-share is more targeted, sharing weights only from poorly performing experts.
The framework provides theoretical "regret bounds," ensuring the cumulative average loss remains close to that of the best possible sequence of experts, even with the handicap of delayed feedback.
Experimental Setup
The approach was evaluated using the Numenta Anomaly Benchmark (NAB), comprising 58 streaming data sources ranging from AWS server metrics to industrial sensors. The ensemble utilized 15 existing unsupervised anomaly detection algorithms from the NAB repository as base experts.
Results
The ensemble methods significantly outperformed standalone competitors on the NAB leaderboard across various metrics and delay scenarios.
As shown in the AUC results (Figure 3), the best individual model (randomCutForest) achieved an AUC of 0.616. In contrast, the authors' Fixed-share approach reached an AUC of 0.998 with minimal delay and maintained a strong 0.893 even with a 100-step delay.
Similar dominance was observed in other metrics (Figure 1), where ensemble methods consistently doubled or tripled the performance of standalone algorithms. The system's ability to maintain high accuracy despite delays of up to 100 data points demonstrates its practical utility in real-world monitoring scenarios.
Discussion and Conclusion
The "superexpert" concept proves that allowing an ensemble to switch between models dynamically provides superior performance over static selection. Beyond accuracy, the method offers interpretability; operators can monitor expert weights to understand which algorithms are currently trusted and why.
While the Fixed-share method is highly flexible, Variable-share is currently limited to bounded loss functions. Future work may focus on optimizing the switching rate ($\alpha$) and reducing the computational overhead of running multiple experts in parallel, which remains a concern for resource-constrained edge devices.
Summary updated