Introduction
Federated Learning (FL) has emerged as a powerful paradigm for collaborative machine learning, allowing multiple parties to train models without centralizing raw data. However, this decentralized approach introduces two critical vulnerabilities: Byzantine attacks, where malicious clients upload poisoned updates to degrade model performance, and privacy leakage, where a curious server can reconstruct private data from shared gradients.
Existing solutions that attempt to address both issues simultaneously often suffer from extreme computational and communication overhead. Performing complex statistical operations, such as distance calculations or median finding, within encrypted Secure Multi-Party Computation (SMPC) environments typically creates massive bottlenecks when dealing with high-dimensional model parameters.
The FLURP Framework
The authors introduce FLURP (Federated Learning with Low-Dimensional Update Representation and Proximity-Based Defense), a framework designed to provide robust defense against poisoning while maintaining strict privacy with minimal overhead. FLURP achieves this through three primary innovations:
- LinfSample (Low-Dimensional Update Representation): Instead of processing millions of parameters, clients generate a "digest" of their updates. By applying a sliding window and calculating the $l_\infty$ norm (maximum absolute value) for each segment, they create a Low-dimensional Update Representation (LUR). This LUR acts as a statistical signature that is sensitive enough to detect malicious deviations without the bulk of the full gradient.
- Proximity-Based Defense: In the encrypted domain, servers calculate the distance between these LURs. Benign updates naturally cluster together, while poisoned updates—even those designed to mimic benign mean and variance—appear as outliers. Updates with a sufficient number of "neighbors" are deemed benign and included in the global model.
- Optimized SMPC Protocols: To facilitate these calculations privately, the framework utilizes custom protocols like
packedCompareandmulRowQuickSelect, allowing servers to find medians and compare values across multiple clients efficiently.
Technical Efficiency and Privacy
A core challenge in SMPC is the round complexity of comparisons. Standard protocols often scale linearly with the number of clients ($n$). FLURP’s packedCompare protocol optimizes this process, significantly reducing the communication rounds required for proximity calculations.
As shown in the complexity analysis, while communication bits remain comparable, the round complexity for $n$ pairs of arithmetic shares is reduced from $n \log l$ to $\log l$. This optimization is a key driver behind the framework's real-world performance.
Experimental Results
The researchers evaluated FLURP using standard datasets including CIFAR-10 (ResNet10), ImageNet-12 (MobileNet-V2), and AgNews (Bi-LSTM). The results demonstrate a dramatic improvement over traditional encrypted defenses:
- Communication Efficiency: For CIFAR-10, the data required to compute the shared distance matrix was slashed from 14.2 GB to just 3.5 MB.
- Latency Reduction: Server-side computation time for the distance matrix dropped from 726 seconds to a mere 0.4 seconds, representing a 1,000x speedup.
- Adversarial Robustness: Even when 40% of clients were malicious, FLURP maintained high accuracy (e.g., ~88% on AgNews). It proved particularly effective against sophisticated attacks like "MinMax" and "ALIE," as well as backdoor attacks in text datasets where other defenses often fail.
Conclusion and Future Directions
FLURP demonstrates that high-dimensional model updates are not required for effective outlier detection in Federated Learning. By moving the "heavy lifting" of defense to low-dimensional digests, the framework provides a scalable path for secure and private collaborative learning.
Future work aims to address challenges in highly non-IID (Independent and Identically Distributed) data environments, where unique but honest client updates might be mistaken for outliers. Additionally, the team plans to move the LinfSample process itself into the SMPC protocol to prevent sophisticated attackers from forging benign-looking LURs while sending poisoned full updates.
Summary updated