CoPur: Certifiably Robust Collaborative Inference via Feature Purification

Jing Liu, Chulin Xie, Oluwasanmi O Koyejo, Bo LiNeurIPS 2022 · OpenReview

Introduction

Collaborative inference—where multiple sensors or agents send data to a central hub—is a powerful paradigm for modern AI, but it is highly vulnerable to "bad actors" providing malicious or missing information. CoPur (Feature Purification) is a novel framework that uses block-sparse optimization and manifold learning to filter out these attacks, providing the first certifiable robustness guarantees for distributed AI systems.

In domains like the Internet of Things (IoT) and Vertical Federated Learning (VFL), collaborative inference is increasingly common. Imagine a smart health system: your watch sends heart rate data, your phone sends GPS data, and a smart ring sends sleep metrics to a "Fusion Center" to predict your overall health status.

However, this setup presents a significant security vulnerability. If just one sensor is compromised, it can send "adversarial" features—carefully crafted noise designed to trick the central model into making an incorrect prediction. Standard defenses typically assume the attacker only makes tiny changes (small $\ell_p$-norm perturbations). In collaborative settings, a hacked sensor can send features with infinitely large magnitudes or simply drop offline entirely (missing-feature attacks). Existing tools like Randomized Smoothing or simple Manifold Projection often fail to handle these structured, large-scale corruptions.

Overview of the collaborative inference threat model and the CoPur feature purification process.

The CoPur Methodology

The authors introduce CoPur, which treats incoming data as a corrupted signal to be "purified" before the final model processes it. The framework relies on two main pillars:

  1. Manifold Learning: The authors assume that even if sensors collect different types of data, there is underlying redundancy across the features. They use an AutoEncoder trained on clean data to learn the "manifold"—the low-dimensional subspace where valid, uncorrupted feature combinations naturally reside.
  2. Block-Sparse Decomposition: When the Fusion Center receives a feature vector, CoPur decomposes it into two distinct parts: a "clean" vector that lies on the learned manifold, and a "corruption" vector. Because only a fraction of agents are typically compromised at any given time, the corruption vector is block-sparse (most blocks of data are clean/zero-error, but a few specific blocks are heavily corrupted).

To make this computationally feasible, the authors relax the problem into an $\ell_{2,1}$-norm minimization. Instead of merely "snapping" a corrupted point to the nearest spot on the manifold (which fails if the attack magnitude is massive), CoPur explicitly identifies which blocks are corrupted, isolates them, and reconstructs the true features using the clean data from the remaining agents.

Key Insights and Theoretical Guarantees

  • Structure Over Magnitude: A core insight of CoPur is that the structure of the attack (identifying which agent is lying) is more important than the size of the lie. By modeling attacks as block-sparse, CoPur can recover the original data even if the attacker sends values that are orders of magnitude larger than normal.
  • Certifiable Recovery: The paper provides rigorous theoretical proofs showing that under specific conditions related to the "leverage constant" of the feature space, CoPur is guaranteed to recover the true features exactly or within a stable, bounded margin.
  • The Feature-Flipping Attack: To test their system, the authors proposed a new, simple "untargeted" attack where a malicious agent flips the sign of its features and amplifies them. While this simple trick easily breaks most state-of-the-art defenses, CoPur remains highly resilient.

Experimental Results

The researchers evaluated CoPur on two diverse datasets: ExtraSensory (predicting user behavior from 10 different phone/watch sensors) and NUS-WIDE (a web image and text dataset split among four agents). CoPur significantly outperformed existing defenses like Randomized Smoothing, Adversarial Training, and Randomized Ablation:

  • Robust Accuracy: On the ExtraSensory dataset, CoPur maintained 79.0% accuracy under a targeted attack, while the unsecured model dropped to 40.4% and other defenses hovered around 50-60%.
  • Resilience to Scale: In tests where multiple agents were malicious, CoPur's accuracy stayed remarkably high even when 8 out of 10 agents were performing attacks. In contrast, traditional adversarial training collapsed once more than two agents were compromised.
  • Combined Attacks: CoPur proved highly effective against "combined" threats where some agents send malicious data while others are simultaneously offline.

Limitations and Future Work

While powerful, CoPur does have practical limitations. First, it requires a set of strictly uncorrupted data to train the initial AutoEncoder; if the training set itself is poisoned, the "purification" will map to a flawed manifold. Second, solving an optimization problem during the inference phase adds latency compared to a simple forward pass of a neural network. Future directions include extending the work to more complex, non-linear manifolds and finding more efficient ways to calculate the leverage constants that define a system's exact corruption tolerance.

Summary updated

Abstract

Collaborative inference leverages diverse features provided by different agents (e.g., sensors) for more accurate inference. A common setup is where each agent sends its embedded features instead of the raw data to the Fusion Center (FC) for joint prediction. In this setting, we consider the inference-time attacks when a small fraction of agents are compromised. The compromised agent either does not send embedded features to the FC, or sends arbitrarily embedded features. To address this, we propose a certifiably robust COllaborative inference framework via feature PURification (CoPur), by leveraging the block-sparse nature of adversarial perturbations on the feature vector, as well as exploring the underlying redundancy across the embedded features (by assuming the overall features lie on an underlying lower dimensional manifold). We theoretically show that the proposed feature purification method can robustly recover the true feature vector, despite adversarial corruptions and/or incomplete observations. We also propose and test an untargeted distributed feature-flipping attack, which is agnostic to the model, training data, label, as well as the features held by other agents, and is shown to be effective in attacking state-of-the-art defenses. Experiments on ExtraSensory and NUS-WIDE datasets show that CoPur significantly outperforms existing defenses in terms of robustness against targeted and untargeted adversarial attacks.
BibTeX
@inproceedings{liu2022copur,
  author = {Jing Liu and Chulin Xie and Oluwasanmi O Koyejo and Bo Li},
  title = {{CoPur: Certifiably Robust Collaborative Inference via Feature Purification}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=r5rzV51GZx},
}
Figure