A Framework for Verifiable and Auditable Federated Anomaly Detection

Gabriele Santin, Inna Skarbovsky, Fabiana Fournier, Bruno Lepri

Abstract

This paper presents a decentralized Federated Learning (FL) framework designed for anomaly detection in high-stakes environments like finance and healthcare. By utilizing Random Forest ensembles and a novel "FIT-SHARE-GET-CROP" cycle, the system enables organizations to collaboratively train models without sharing raw data. Integration with Hyperledger Fabric provides a verifiable audit trail of all operations. Experimental results on credit card fraud detection show significant performance gains, particularly for nodes with limited local anomaly data, while ensuring model generalization and process transparency.

Introduction

Anomaly detection—identifying rare but critical events like fraudulent transactions or medical emergencies—is a vital task for modern organizations. However, developing robust models often requires large, diverse datasets that individual entities may lack. While pooling data seems like a logical solution, legal, privacy, and competitive barriers frequently prevent direct data sharing. Federated Learning (FL) offers a path forward by allowing models to be trained across decentralized nodes without exchanging raw data.

Traditional FL architectures often rely on a central coordinator, which introduces a single point of failure and potential trust issues. Furthermore, in high-stakes applications, accuracy alone is insufficient; stakeholders require auditability and verifiability. This paper proposes a decentralized FL framework that utilizes Random Forests and Blockchain technology to enable collaborative anomaly detection that is both private and fully auditable.

System Architecture

The proposed framework is built on a decentralized architecture where machine learning models are treated as collections of "atomic" building blocks—specifically, individual decision trees within a Random Forest ensemble.

High-level architecture of the federated learning framework, showing the interaction between the secure execution environment and distributed learning nodes.

As illustrated in Figure 1, the system comprises a BC Secure Execution Environment that manages identity, artifact storage, and usage audits. Data scientists publish algorithm images to a registry, which are then deployed across multiple learning nodes. Each node operates on its own local organizational data, ensuring that sensitive information never leaves the premises while contributing to the global model.

The FIT-SHARE-GET-CROP Cycle

The core of the collaborative process is a four-stage cycle designed to evolve the global model while maintaining local efficiency:

  1. FIT: Each node trains new decision trees on its local dataset.
  2. SHARE: Nodes identify their most effective trees and share them with peers.
  3. GET: Nodes receive trees from their neighbors and integrate them into their local forests.
  4. CROP: To prevent model bloat, nodes prune their ensembles, retaining only the highest-quality trees.

A key innovation in this cycle is the use of a tree kernel for unsupervised ranking. By measuring the structural similarity between trees using a Gaussian Process approach, the system can select a diverse and representative subset of trees. This method avoids the bias inherent in ranking based on local, potentially skewed labels, ensuring that the shared knowledge is globally relevant and prevents "knowledge silos."

Blockchain-Enabled Auditability

To ensure the integrity and transparency of the federated process, the framework integrates Hyperledger Fabric. The blockchain serves as a secure, immutable ledger that records every significant operation within the network.

Detailed view of the blockchain shared ledger and its orchestration of the learning process via specialized chaincodes.

Figure 2 details the interaction between various chaincodes within the BC Shared Ledger. The image chaincode stores algorithm metadata, while the learning process orchestration chaincode manages the overall workflow. The execution record chaincode provides a verifiable audit trail of all computations, and the model chaincode stores the final, completed models. This structure ensures that stakeholders can verify which node contributed specific logic and confirm that the algorithms were executed as intended without malicious interference.

Experimental Evaluation and Results

The framework was evaluated using a standard credit card fraud detection benchmark containing 284,807 transactions, where only 0.17% were labeled as fraudulent. To simulate a realistic decentralized environment, the authors established a network of 20 agents with highly unbalanced data distributions.

The results demonstrated that collaboration is essential for "data-poor" nodes. For instance, "Node 2," which initially had zero fraud examples in its local set, was unable to detect any anomalies in isolation. After joining the federation, its precision skyrocketed to 0.9. Overall, the federation led to a mean improvement in precision and recall of 0.1 to 0.2 across the network.

Furthermore, the study found that the federated models generalized better to unseen data than those trained in isolation. While local accuracy sometimes decreased slightly, the ability to detect universal patterns of anomaly significantly improved, proving the framework's effectiveness in mitigating overfitting to local data quirks.

Conclusion

This framework provides a robust solution for organizations needing to collaborate on anomaly detection without compromising data privacy or trust. By combining ensemble learning with blockchain-backed auditability, it creates a verifiable environment for high-stakes machine learning. Future work will explore extending this logic to other ensemble methods like Gradient Boosting and optimizing network topologies to further enhance learning speed and security.

Summary updated

Abstract

Collaborative and Federated Leaning are emerging approaches to manage cooperation between a group of agents for the solution of Machine Learning tasks, with the goal of improving each agent’s performance without disclosing any data. In this paper we present a novel algorithmic architecture that tackle this problem in the particular case of Anomaly Detection (or classification of rare events), a setting where typical applications often comprise data with sensible information, but where the scarcity of anomalous examples encourages collaboration. We show how Random Forests can be used as a tool for the development of accurate classifiers with an effective insight-sharing mechanism that does not break the data integrity. Moreover, we explain how the new architecture can be readily integrated in a blockchain infrastructure to ensure the verifiable and auditable execution of the algorithm. Furthermore, we discuss how this work may set the basis for a more general approach for the design of collaborative ensemble-learning methods beyond the specific task and architecture discussed in this paper.
BibTeX
@article{santin2022framework,
  author = {Gabriele Santin and Inna Skarbovsky and Fabiana Fournier and Bruno Lepri},
  title = {{A Framework for Verifiable and Auditable Federated Anomaly Detection}},
  journal = {IEEE Access},
  year = {2022},
  doi = {10.1109/access.2022.3196391},
}
Figure