Deep Poisoning: Towards Robust Image Data Sharing against Visual Disclosure

Hao Guo, Brian Dolhansky, Eric Hsin, Phong Dinh, Cristian Canton-Ferrer, Song Wang

Introduction

In the era of deep learning, data is the most valuable asset. However, many critical computer vision tasks—such as identifying child exploitation imagery (CEI) or terrorist propaganda—involve highly sensitive or illegal content. This creates a "data island" problem: individual organizations often lack sufficient data to train robust models independently, yet legal and ethical barriers prevent them from sharing raw images to build a unified dataset.

Existing privacy-preserving methods, such as anonymization or standard encryption, often fall short. If organizations share an obfuscation method to maintain a common feature space, a malicious actor could potentially reverse the process to reconstruct the original images. This paper introduces Deep Poisoning, a technique that enables collaborative model training without ever revealing sensitive visual content, providing a "reconstruction-proof" guarantee.

The Deep Poisoning Framework

The authors propose a three-step regime centered around a private Deep Poisoning Module (DPM). This module is designed to mangle the spatial information of an image to the point of being unrecognizable while retaining the specific signals a classifier needs for identification.

  1. Initial Pre-training: A standard CNN (e.g., ResNet) is pre-trained on a non-sensitive dataset. The model is split into a featurizer ($\phi_1$) for high-level extraction and a decider ($\phi_2$) for final classification.
  2. DPM Training: Each entity creates a private DPM ($P$) consisting of several convolutional layers, inserted between the featurizer and decider. The DPM is trained using adversarial loss to balance two goals: Classification Equivalence (maintaining accuracy) and Reconstruction Disparity (preventing image recovery).
  3. Model Refinement: Entities keep their DPMs private but share the "poisoned" features. Because these features are functionally equivalent for the task, other entities can use them to refine their own deciders without ever seeing the source images.
Overview of the Deep Poisoning framework, illustrating (a) Image Data Sharing with DPMs, (b) Model Refinement, and (c) Inference.

Experimental Evaluation

The approach was evaluated using ImageNet (split into sensitive and public sets) and CelebA (for face attribute recognition).

Key Results

  • High Utility, Low Disclosure: On ResNet50, top-1 accuracy on ImageNet only dropped from 79.39% to 78.88%. Meanwhile, the Structural Similarity Index (SSIM)—where 1.0 is a perfect match—dropped to a negligible 0.007, meaning reconstructed images were essentially visual noise.
  • The "Small Player" Boost: In simulations where one entity had only 1% of the training data, its standalone model achieved only 13.5% accuracy. By incorporating poisoned features shared by three collaborators, its accuracy jumped to 61.6%.
  • Robustness against Attacks: The DPM successfully defended against five different architectures designed specifically to "attack" and reconstruct images from features, maintaining high reconstruction error across the board.

Key Insights

  • Privacy through Secrecy: The core innovation is that the DPM remains private. Without the specific weights of the DPM, a collaborator cannot build a successful "de-poisoning" reconstructor, even if the base featurizer is public.
  • Functional Equivalence: The study proves that visual clarity is not a prerequisite for mathematical utility. Spatial information can be distorted beyond recognition while preserving the features necessary for categorical classification.
  • Asynchronous Collaboration: Unlike Federated Learning, which requires complex infrastructure for simultaneous training, Deep Poisoning allows for asynchronous data sharing through simple feature file exchanges.

Limitations and Future Work

While promising, the current study assumes IID (Independent and Identically Distributed) data across entities. In real-world scenarios, data distributions may vary significantly. Additionally, DPMs are currently task-specific; future work will explore how poisoned features perform across different tasks like object detection or segmentation without extensive retraining.

Summary updated

Abstract

Due to respectively limited training data, different entities addressing the same vision task based on certain sensitive images may not train a robust deep network. This paper introduces a new vision task where various entities share task-specific image data to enlarge each other’s training data volume without visually disclosing sensitive contents (e.g. illegal images). Then, we present a new structure-based training regime to enable different entities learn task-specific and reconstruction-proof image representations for image data sharing. Specifically, each entity learns a private Deep Poisoning Module (DPM) and insert it to a pre-trained deep network, which is designed to perform the specific vision task. The DPM deliberately poisons convolutional image features to prevent image reconstructions, while ensuring that the altered image data is functionally equivalent to the non-poisoned data for the specific vision task. Given this equivalence, the poisoned features shared from one entity could be used by another entity for further model refinement. Experimental results on image classification prove the efficacy of the proposed method.
BibTeX
@inproceedings{guo2021deep,
  author = {Hao Guo and Brian Dolhansky and Eric Hsin and Phong Dinh and Cristian Canton-Ferrer and Song Wang},
  title = {{Deep Poisoning: Towards Robust Image Data Sharing against Visual Disclosure}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
  doi = {10.1109/wacv48630.2021.00073},
}
Figure