FedLAD: A Linear Algebra Based Data Poisoning Defence for Federated Learning

Qi Xiong, Hai Dong, Nasrin Sohrabi, Zahir TariarXiv 2025 · DOI · arXiv

Introduction

Federated Learning (FL) enables collaborative machine learning model training across multiple edge devices without the need to share raw, private data. However, this decentralized architecture is highly vulnerable to Sybil attacks and targeted data poisoning, such as label flipping. In these scenarios, malicious nodes submit corrupted model updates to the central server to degrade the global model's performance.

Existing defenses, typically based on clustering or robust aggregation (e.g., median or trimmed mean), share a critical vulnerability: they fail when malicious nodes constitute more than 50% of the network. In such high-compromise environments, malicious updates become the statistical majority, effectively "poisoning" the global model. This paper introduces FedLAD (Linear Algebra-based Detection), a novel defense mechanism that leverages Row-Reduced Echelon Form (RREF) to identify and filter out malicious updates, maintaining robustness even when up to 80% of the network is compromised.

Methodology: Linear Algebra for Model Filtering

FedLAD shifts the defense paradigm from statistical distance metrics to linear algebraic independence. The core insight is that malicious actors often rely on redundancy—submitting multiple similar, corrupted updates—to overwhelm the server.

The FedLAD process involves four primary steps:

  1. Flattening: The central server flattens the weights of all incoming local models into 1D vectors and stacks them into a single matrix.
  2. RREF Calculation: The server computes the Row-Reduced Echelon Form (RREF) of this matrix. This identifies the "pivot rows," which represent the minimal set of linearly independent vectors that form the basis of the update space.
  3. Filtering: Models that are linearly dependent on others are discarded. The authors demonstrate that these dependent vectors add no new representational capacity and often represent redundant noise or adversarial replicas.
  4. Aggregation: Standard Federated Averaging (FedAVG) is performed exclusively on the independent pivot models.

To address the computational complexity of RREF (Gaussian Elimination) on large neural network weight matrices, the authors developed a parallelized divide-and-conquer algorithm. This approach splits the matrix into sub-matrices, processes them simultaneously across multiple CPUs, and recombines the results, significantly reducing processing time.

Experimental Evaluation

FedLAD was evaluated across diverse tasks and datasets to ensure its generalizability:

  • Computer Vision: CIFAR10 and CIFAR100 using ResNet18.
  • Natural Language Processing: AG_NEWS using a 3-layer neural network.
  • Data Distribution: To simulate realistic FL environments, data was distributed in a Non-IID (non-independent and identically distributed) manner using a Dirichlet distribution.

Key Results and Insights

The primary strength of FedLAD is its ability to break the "50% barrier." Because it relies on mathematical independence rather than majority voting, it remains effective regardless of the ratio of malicious nodes.

Comparison of ML Accuracy across different attacker ratios for various datasets. FedLAD maintains higher accuracy even as the attacker ratio exceeds 0.5.

Key findings include:

  • Robustness to High Attacker Ratios: FedLAD maintained a significantly lower Attack Success Rate (ASR) and higher Model Accuracy (MA) compared to baselines like Sherpa, CONTRA, Median, Trimmed Mean, and Krum. While baselines failed once attackers exceeded 50%, FedLAD remained resilient up to 70-80% compromise.
  • Computational Efficiency: The parallel RREF algorithm provided substantial speedups. For CIFAR10, processing time dropped from ~245 minutes (serial) to ~63 minutes (parallel).

Conclusion and Future Directions

FedLAD demonstrates that redundancy in model updates is a significant vulnerability that can be mitigated through linear algebra. While the method is highly effective at filtering attacks, model accuracy naturally declines at extreme compromise levels (e.g., 80%) due to the scarcity of benign training data. Future research could explore combining RREF-based filtering with other robust aggregation schemes to further enhance performance in adversarial environments.

Summary updated

Abstract

Sybil attacks pose a significant threat to federated learning, as malicious nodes can collaborate and gain a majority, thereby overwhelming the system. Therefore, it is essential to develop countermeasures that ensure the security of federated learning environments. We present a novel defence method against targeted data poisoning, which is one of the types of Sybil attacks, called Linear Algebra-based Detection (FedLAD). Unlike existing approaches, such as clustering and robust training, which struggle in situations where malicious nodes dominate, FedLAD models the federated learning aggregation process as a linear problem, transforming it into a linear algebra optimisation challenge. This method identifies potential attacks by extracting the independent linear combinations from the original linear combinations, effectively filtering out redundant and malicious elements. Extensive experimental evaluations demonstrate the effectiveness of FedLAD compared to five well-established defence methods: Sherpa, CONTRA, Median, Trimmed Mean, and Krum. Using tasks from both image classification and natural language processing, our experiments confirm that FedLAD is robust and not dependent on specific application settings. The results indicate that FedLAD effectively protects federated learning systems across a broad spectrum of malicious node ratios. Compared to baseline defence methods, FedLAD maintains a low attack success rate for malicious nodes when their ratio ranges from 0.2 to 0.8. Additionally, it preserves high model accuracy when the malicious node ratio is between 0.2 and 0.5. These findings underscore FedLAD's potential to enhance both the reliability and performance of federated learning systems in the face of data poisoning attacks.
BibTeX
@misc{xiong2025fedlad,
  author = {Qi Xiong and Hai Dong and Nasrin Sohrabi and Zahir Tari},
  title = {{FedLAD: A Linear Algebra Based Data Poisoning Defence for Federated Learning}},
  year = {2025},
  eprint = {2508.02136},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.02136},
}
Figure