Deep Anomaly Detection and Search via Reinforcement Learning (Student Abstract)

Chao Chen, Dawei Wang, Feng Mao, Zongzhang Zhang, Yang YuAAAI 2023 · DOI

Introduction

Anomaly Detection (AD) aims to identify rare events or items that deviate significantly from the norm. In practical applications, researchers often face a semi-supervised challenge: a small set of labeled anomalies paired with a vast pool of unlabeled data. Traditional AD methods frequently suffer from rigid structural assumptions (e.g., clustering) and extreme sensitivity to "contamination"—hidden anomalies within the unlabeled training set that degrade model performance.

The DADS Framework

To address these challenges, the authors introduce Deep Anomaly Detection and Search (DADS). This framework reformulates anomaly detection as a sequential decision-making process, leveraging Reinforcement Learning (RL) to proactively "hunt" for unknown anomaly classes.

The DADS architecture: An RL agent interacts with a hierarchical dataset environment to identify and confirm anomalies.

The core of DADS is a Soft Actor-Critic (SAC) agent. SAC's entropy-maximization objective encourages the agent to explore rare or unseen data points. The training environment is structured into three hierarchical pools:

  1. Unlabeled Dataset ($U$): The initial, potentially contaminated data source.
  2. Temporary Dataset ($T$): A buffer for data points suspected of being anomalous.
  3. Anomaly Dataset ($A$): A collection of confirmed anomalies used for model refinement.

A data point is only moved to the Anomaly pool if it is flagged consistently across multiple iterations, meeting a confidence threshold ($TH_{conf}$). This mechanism acts as a filter against noise and one-off classification errors. The agent's learning is guided by a dual reward system: a supervised reward for known anomalies and an unsupervised reward (derived from an Isolation Forest) to capture the general data distribution.

Experimental Evaluation

DADS was tested on three multi-dimensional tabular datasets: multi_annthyroid, multi_cardio, and multi_har. To simulate realistic conditions, only 10% of anomalies were labeled as "known," forcing the model to discover the remaining 90%. The researchers specifically tested robustness by varying the contamination ratio of the unlabeled set from 0% to 10%.

Key Results and Robustness

DADS consistently outperformed established baselines, including supervised (XGBoost), unsupervised (Isolation Forest), and semi-supervised methods (DeepSAD, DevNet).

Performance comparison across varying contamination ratios. DADS maintains high AUC-ROC stability compared to declining baselines.

The most significant finding is DADS's resistance to contamination. While the performance of competing semi-supervised models like DevNet and DeepSAD plummeted as hidden anomalies were added to the training set, DADS maintained remarkably stable AUC-ROC scores. This stability stems from the RL agent's exploratory nature and the hierarchical confirmation process, which allows the model to identify and leverage "unknown" anomalies rather than being confused by them.

Conclusion and Future Directions

DADS demonstrates that framing anomaly detection as an RL search problem effectively overcomes the limitations of static semi-supervised models. While highly effective on tabular data, future research is needed to scale this hierarchical search to high-dimensional domains like image or video data. Additionally, the computational overhead of the SAC agent and ensemble sampling remains a factor for real-time applications.

Summary updated

BibTeX
@inproceedings{chen2023deep,
  author = {Chao Chen and Dawei Wang and Feng Mao and Zongzhang Zhang and Yang Yu},
  title = {{Deep Anomaly Detection and Search via Reinforcement Learning (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i13.26950},
}
Figure