Introduction
Anomaly detection (AD) is a critical task in identifying rare, high-stakes events such as medical emergencies, financial fraud, or cybersecurity breaches. In many real-world scenarios, the data required to train robust AD models is sensitive and siloed across multiple organizations—for instance, patient records in different hospitals or transaction logs in separate banks. Federated Learning (FL) has emerged as a promising solution, allowing models to learn from decentralized data without requiring the raw records to leave their local environments.
Despite the potential of FL, the field of Federated Anomaly Detection (FAD) has historically lacked standardization. Most existing research focuses on image or text data, leaving a significant gap for tabular data, which remains the most common format in industrial and medical applications. Furthermore, many existing benchmarks "pollute" training sets with anomalies, failing to reflect true unsupervised learning where the model must learn the distribution of "normal" data in isolation. To address these challenges, we introduce FedAD-Bench, the first unified benchmark designed specifically for unsupervised deep learning anomaly detection on tabular data in federated environments.
The FedAD-Bench Framework
FedAD-Bench standardizes data splitting, evaluation metrics, and federated configurations to provide a rigorous testing ground for FAD algorithms.
True Unsupervised Learning
Unlike traditional benchmarks that may include labeled anomalies during training, FedAD-Bench implements a strict unsupervised protocol. All anomalies are removed from the training set, forcing models to learn a representation of "normality." During testing, the models must identify outliers based on their deviation from this learned distribution.
Model Selection and Aggregation
The benchmark adapts five state-of-the-art (SOTA) deep learning models for the federated setting:
- Deep Autoencoder (DAE): Detects anomalies based on reconstruction error.
- Deep Structured Energy-Based Model (DSEBM): Utilizes energy scores to identify low-probability samples.
- DeepSVDD: Maps data into a minimum-volume hypersphere to isolate outliers.
- NeuTraLAD: Employs contrastive learning and neural transformations.
- Memory-augmented Autoencoder (MemAE): Uses a memory module to store prototypes of normal data.
These models are evaluated using two primary aggregation strategies: FedAvg, the standard federated averaging approach, and FedProx, which introduces a proximal term to penalize local models that stray too far from the global average, helping to mitigate client drift.
Experimental Setup
FedAD-Bench utilizes four diverse tabular datasets across two critical domains:
- Medical: Arrhythmia (cardiac pattern analysis) and Thyroid (disease classification).
- Cybersecurity: KDDCUP10 and NSL-KDD (network intrusion detection).
Key Insights and Results
The "Regularization Effect" of Federated Learning
One of the most surprising findings of the study is that federated training is not always a compromise. In 4 out of 20 experimental scenarios, the federated model actually outperformed its centralized counterpart. This suggests that the act of averaging local models can serve as a powerful regularizer, preventing the global model from overfitting to the specific noise or quirks of any single local dataset.
Architecture and FL-Friendliness
The benchmark reveals that simpler architectures, such as the Deep Autoencoder (DAE), are often more "FL-friendly." In contrast, complex models like MemAE struggle in federated settings. This is largely because MemAE relies on internal heuristics—specifically its memory module—which are not currently synchronized during the standard aggregation process. This loss of "learned prototypes" during averaging significantly degrades performance.
Metric Sensitivity: AUROC vs. AUPR
The authors argue that the Area Under the Receiver Operating Characteristic (AUROC) can be overly optimistic in anomaly detection because it gives equal weight to the majority (normal) class. In the context of highly imbalanced data, the Area Under the Precision-Recall curve (AUPR) provides a more robust and realistic measure of a model's ability to identify rare anomalies.
Scalability and Data Homogeneity
Performance generally decreases as the number of clients increases, particularly on diverse datasets like Thyroid. As data is spread thinner across more clients, each local model has fewer samples to learn from, leading to poorer generalization. However, on highly uniform (homogeneous) datasets like KDDCUP10, the performance remains relatively stable even as the client count grows. The study also found that FedProx was particularly effective at bridging the gap between federated and centralized performance in these homogeneous environments.
Limitations and Future Work
While FedAD-Bench provides a much-needed foundation, several challenges remain:
- Heuristic Synchronization: Future work must develop specialized aggregation methods for models with internal states, such as MemAE’s memory modules.
- Dataset Diversity: Some current datasets are insufficiently challenging, leading to near-perfect scores that do not adequately differentiate between SOTA models.
- Non-IID Data: This initial benchmark focused on uniform data distribution. Exploring the impact of highly skewed (Non-IID) data across clients is a critical next step for the evolution of FedAD-Bench.
Conclusion
FedAD-Bench brings necessary order to the "Wild West" of federated anomaly detection. By providing a standardized framework and revealing the unique regularization benefits of federated aggregation, this work paves the way for more robust and privacy-preserving anomaly detection in real-world tabular applications.
Summary updated