Introduction
Federated Learning (FL) enables collaborative machine learning model training across multiple edge devices without the need to share raw, private data. However, this decentralized architecture is highly vulnerable to Sybil attacks and targeted data poisoning, such as label flipping. In these scenarios, malicious nodes submit corrupted model updates to the central server to degrade the global model's performance.
Existing defenses, typically based on clustering or robust aggregation (e.g., median or trimmed mean), share a critical vulnerability: they fail when malicious nodes constitute more than 50% of the network. In such high-compromise environments, malicious updates become the statistical majority, effectively "poisoning" the global model. This paper introduces FedLAD (Linear Algebra-based Detection), a novel defense mechanism that leverages Row-Reduced Echelon Form (RREF) to identify and filter out malicious updates, maintaining robustness even when up to 80% of the network is compromised.
Methodology: Linear Algebra for Model Filtering
FedLAD shifts the defense paradigm from statistical distance metrics to linear algebraic independence. The core insight is that malicious actors often rely on redundancy—submitting multiple similar, corrupted updates—to overwhelm the server.
The FedLAD process involves four primary steps:
- Flattening: The central server flattens the weights of all incoming local models into 1D vectors and stacks them into a single matrix.
- RREF Calculation: The server computes the Row-Reduced Echelon Form (RREF) of this matrix. This identifies the "pivot rows," which represent the minimal set of linearly independent vectors that form the basis of the update space.
- Filtering: Models that are linearly dependent on others are discarded. The authors demonstrate that these dependent vectors add no new representational capacity and often represent redundant noise or adversarial replicas.
- Aggregation: Standard Federated Averaging (FedAVG) is performed exclusively on the independent pivot models.
To address the computational complexity of RREF (Gaussian Elimination) on large neural network weight matrices, the authors developed a parallelized divide-and-conquer algorithm. This approach splits the matrix into sub-matrices, processes them simultaneously across multiple CPUs, and recombines the results, significantly reducing processing time.
Experimental Evaluation
FedLAD was evaluated across diverse tasks and datasets to ensure its generalizability:
- Computer Vision: CIFAR10 and CIFAR100 using ResNet18.
- Natural Language Processing: AG_NEWS using a 3-layer neural network.
- Data Distribution: To simulate realistic FL environments, data was distributed in a Non-IID (non-independent and identically distributed) manner using a Dirichlet distribution.
Key Results and Insights
The primary strength of FedLAD is its ability to break the "50% barrier." Because it relies on mathematical independence rather than majority voting, it remains effective regardless of the ratio of malicious nodes.
Key findings include:
- Robustness to High Attacker Ratios: FedLAD maintained a significantly lower Attack Success Rate (ASR) and higher Model Accuracy (MA) compared to baselines like Sherpa, CONTRA, Median, Trimmed Mean, and Krum. While baselines failed once attackers exceeded 50%, FedLAD remained resilient up to 70-80% compromise.
- Computational Efficiency: The parallel RREF algorithm provided substantial speedups. For CIFAR10, processing time dropped from ~245 minutes (serial) to ~63 minutes (parallel).
Conclusion and Future Directions
FedLAD demonstrates that redundancy in model updates is a significant vulnerability that can be mitigated through linear algebra. While the method is highly effective at filtering attacks, model accuracy naturally declines at extreme compromise levels (e.g., 80%) due to the scarcity of benign training data. Future research could explore combining RREF-based filtering with other robust aggregation schemes to further enhance performance in adversarial environments.
Summary updated