Introduction
Collaborative inference—where multiple sensors or agents send data to a central hub—is a powerful paradigm for modern AI, but it is highly vulnerable to "bad actors" providing malicious or missing information. CoPur (Feature Purification) is a novel framework that uses block-sparse optimization and manifold learning to filter out these attacks, providing the first certifiable robustness guarantees for distributed AI systems.
In domains like the Internet of Things (IoT) and Vertical Federated Learning (VFL), collaborative inference is increasingly common. Imagine a smart health system: your watch sends heart rate data, your phone sends GPS data, and a smart ring sends sleep metrics to a "Fusion Center" to predict your overall health status.
However, this setup presents a significant security vulnerability. If just one sensor is compromised, it can send "adversarial" features—carefully crafted noise designed to trick the central model into making an incorrect prediction. Standard defenses typically assume the attacker only makes tiny changes (small $\ell_p$-norm perturbations). In collaborative settings, a hacked sensor can send features with infinitely large magnitudes or simply drop offline entirely (missing-feature attacks). Existing tools like Randomized Smoothing or simple Manifold Projection often fail to handle these structured, large-scale corruptions.
The CoPur Methodology
The authors introduce CoPur, which treats incoming data as a corrupted signal to be "purified" before the final model processes it. The framework relies on two main pillars:
- Manifold Learning: The authors assume that even if sensors collect different types of data, there is underlying redundancy across the features. They use an AutoEncoder trained on clean data to learn the "manifold"—the low-dimensional subspace where valid, uncorrupted feature combinations naturally reside.
- Block-Sparse Decomposition: When the Fusion Center receives a feature vector, CoPur decomposes it into two distinct parts: a "clean" vector that lies on the learned manifold, and a "corruption" vector. Because only a fraction of agents are typically compromised at any given time, the corruption vector is block-sparse (most blocks of data are clean/zero-error, but a few specific blocks are heavily corrupted).
To make this computationally feasible, the authors relax the problem into an $\ell_{2,1}$-norm minimization. Instead of merely "snapping" a corrupted point to the nearest spot on the manifold (which fails if the attack magnitude is massive), CoPur explicitly identifies which blocks are corrupted, isolates them, and reconstructs the true features using the clean data from the remaining agents.
Key Insights and Theoretical Guarantees
- Structure Over Magnitude: A core insight of CoPur is that the structure of the attack (identifying which agent is lying) is more important than the size of the lie. By modeling attacks as block-sparse, CoPur can recover the original data even if the attacker sends values that are orders of magnitude larger than normal.
- Certifiable Recovery: The paper provides rigorous theoretical proofs showing that under specific conditions related to the "leverage constant" of the feature space, CoPur is guaranteed to recover the true features exactly or within a stable, bounded margin.
- The Feature-Flipping Attack: To test their system, the authors proposed a new, simple "untargeted" attack where a malicious agent flips the sign of its features and amplifies them. While this simple trick easily breaks most state-of-the-art defenses, CoPur remains highly resilient.
Experimental Results
The researchers evaluated CoPur on two diverse datasets: ExtraSensory (predicting user behavior from 10 different phone/watch sensors) and NUS-WIDE (a web image and text dataset split among four agents). CoPur significantly outperformed existing defenses like Randomized Smoothing, Adversarial Training, and Randomized Ablation:
- Robust Accuracy: On the ExtraSensory dataset, CoPur maintained 79.0% accuracy under a targeted attack, while the unsecured model dropped to 40.4% and other defenses hovered around 50-60%.
- Resilience to Scale: In tests where multiple agents were malicious, CoPur's accuracy stayed remarkably high even when 8 out of 10 agents were performing attacks. In contrast, traditional adversarial training collapsed once more than two agents were compromised.
- Combined Attacks: CoPur proved highly effective against "combined" threats where some agents send malicious data while others are simultaneously offline.
Limitations and Future Work
While powerful, CoPur does have practical limitations. First, it requires a set of strictly uncorrupted data to train the initial AutoEncoder; if the training set itself is poisoned, the "purification" will map to a flawed manifold. Second, solving an optimization problem during the inference phase adds latency compared to a simple forward pass of a neural network. Future directions include extending the work to more complex, non-linear manifolds and finding more efficient ways to calculate the leverage constants that define a system's exact corruption tolerance.
Summary updated