TL;DR
Deep anomaly detection models are highly susceptible to performance degradation when their training data is contaminated with anomalies. This paper proposes a novel "contradicting training" approach that leverages a small set of labeled anomalies to mitigate this contamination, significantly improving detection accuracy without the need for expensive data cleaning.
What Problem Does This Paper Address?
Anomaly detection is crucial for identifying unusual patterns in vast datasets, with applications ranging from cybersecurity to fraud detection. While deep learning models have shown great promise in this area by automatically learning complex feature representations, they suffer from a critical vulnerability: data contamination. When anomalies are present in the training data, deep models can mistakenly learn them as normal, leading to a drastic reduction in their ability to detect similar anomalies later. Manually cleaning large datasets to remove all anomalies is often prohibitively expensive and time-consuming. This paper addresses the challenge of robust anomaly detection in the presence of contaminated training data, aiming to develop a method that is both effective and data-efficient.
Methods
The authors first establish a theoretical foundation by demonstrating that training a deep anomaly detector implicitly induces a kernel machine. This kernel-based perspective reveals that deep detectors rely on a memory-based mechanism, making them susceptible to misinterpreting contaminated data. They derive an information-theoretic bound that quantifies the performance degradation as a quadratic function of the contamination ratio.
To combat this degradation, they propose a "contradicting training" approach. This method involves training the model on the contaminated dataset as usual (to learn normality) while simultaneously using a small, auxiliary dataset of labeled anomalies. The key innovation lies in how these labeled anomalies are used: instead of simply minimizing their anomaly score, the approach discourages learning them as normal. This is achieved by modifying the loss function to penalize the model for assigning low anomaly scores to these labeled anomalies, effectively forcing it to learn to distinguish them from normal data. This is implemented through a gradient-descent step on the contaminated data and a gradient-ascent step on the labeled anomalies, aiming to amplify the anomaly score for anomalous inputs.
Datasets
The paper evaluates its approach on a variety of datasets:
- Synthetic Data: Two types of random monotonic sequences (increasing for normal, decreasing for anomalous) were synthesized to test the theoretical findings.
- Time-Series Data:
- ECG 5000 dataset (electrocardiograms)
- HAR Human Activity Recognition dataset
- Image Classification Data:
- MNIST dataset
- Fashion-MNIST dataset
The experiments involve intentionally contaminating the training sets with varying percentages of anomalies and using a small number of labeled anomalies for training.
Key Insights and Ideas
- Deep Anomaly Detectors as Implicit Kernel Machines: This theoretical insight explains why contamination is so detrimental. The memory-based nature of kernel methods means that if anomalies are in the memory (training data), they can be mistaken for normal data.
- Quadratic Degradation: The finding that performance degrades quadratically with contamination ratio highlights the severity of the problem and the need for robust solutions. Even small contamination levels can have a significant impact.
- Contradicting Training for Data Efficiency: The core idea of using labeled anomalies not just to learn what's normal, but to actively discourage learning what's anomalous, is a clever way to leverage limited labeled data. This is significantly more practical than cleaning entire datasets.
- The Importance of the Amplification Factor (λ): The paper emphasizes that a simple "ramp loss" (equivalent to their method with λ=1) is insufficient. A larger amplification factor (λ) is crucial for effectively pushing anomaly scores higher for anomalous inputs, especially when contamination is present.
Key Results
- Significant Improvement in Contaminated Data: The proposed "contradicting training" approach consistently outperformed the baseline (training on contaminated data without labeled anomalies) and other compared methods (DeepSAD, DevNet) across various datasets, especially under contamination.
- Data Efficiency: The method showed strong performance even with a small number of labeled anomalies (e.g., 20 or 40), demonstrating its practicality and affordability compared to full dataset cleaning.
- Robustness to Contamination: The synthetic data experiments clearly showed that contamination leads to a sharp drop in performance (AUPRC), but the proposed method with a suitable λ significantly mitigates this decay.
- Outperforming State-of-the-Art: On time-series and image datasets, the approach achieved the best or competitive results, often surpassing methods like DeepSAD and DevNet, particularly when contamination was present.
Limitations and Future Work
- Assumption of Good Neural Network: The paper assumes that a "good neural network" is available. The effectiveness of their approach relies on the underlying network's capacity to learn.
- Correlation of Labeled Anomalies: The method works best when labeled anomalies are correlated with the actual anomalies to be detected. If labeled anomalies are not representative, performance might suffer. The authors caution that labeled data should not be correlated with normal data.
- Future Work: The authors suggest that for detecting truly "difficult unknown anomalies," techniques like domain adaptation might be necessary. They also mention that their approach has been validated on real-world telecommunication signal data, indicating potential for broader application.
Summary updated