Abstract
Redactor is a novel, data-centric defense that protects individuals from privacy-leaking machine learning models by injecting "disinformation" into public datasets. Unlike traditional defenses that require control over the model, Redactor works from the outside, generating synthetic data points that "dilute" a target's personal information and successfully thwart membership inference attacks.
The Challenge of Third-Party Model Privacy
As personal data is increasingly scraped from the web to train massive machine learning (ML) models, individuals face a growing threat: inference attacks. A Membership Inference Attack (MIA), for instance, can determine if specific data was used to train a model, potentially leaking sensitive details about health, finances, or identity.
Existing defenses, such as adding noise to model outputs or using differential privacy during training, share a fatal flaw: they assume the defender owns the model or the training process. In reality, data is often used by third-party platforms beyond individual control. Redactor tackles the challenge of targeted disinformation generation, enabling individuals to protect their specific information with zero access to the victim model or the labeling process.
The Redactor Pipeline
The authors propose Redactor, a four-stage pipeline designed to generate effective disinformation in the input space:
- Surrogate Labeler Training: Since the defender doesn't know how the third-party platform will label data, Redactor trains a diverse ensemble of "surrogate labelers" (e.g., neural nets, SVMs, decision trees) on available public data.
- Probabilistic Decision Boundary (PDB) Creation: Using techniques from data programming (specifically the Snorkel framework), Redactor combines these surrogate labelers into a single probabilistic model. This PDB conservatively estimates the "mental boundary" of a human labeler, identifying areas where data is likely to be classified differently than the target.
- Candidate Generation: To find points close to the target, Redactor uses watermarking (interpolating between the target and a base example) and CTGAN (a Conditional Tabular GAN) to generate realistic synthetic candidates.
- Disinformation Selection: Finally, Redactor selects the candidates that are closest to the target in the input space but fall on the "other side" of the PDB.
By injecting these points into the "unlabeled" web, the defense assumes they will be crawled and used by third-party models, effectively "redacting" the target's influence on the resulting model.
Experimental Evaluation
The study evaluates Redactor across four diverse tabular datasets and one image dataset, including AdultCensus, COMPAS, Epileptic Seizure Recognition (ESR), Diabetes, and MNIST.
Key Results
- Effective Defense: On the AdultCensus dataset, Redactor reduced the target's accuracy in the victim model by ~37% and confidence by ~26%, significantly outperforming baselines.
- Minimal Collateral Damage: While protecting the target, the overall test accuracy of the victim model only dropped by about 2%, making the defense stealthy and unlikely to be detected by the model owner.
- Human-Indistinguishable: In a user study with 33 experts, humans could not reliably distinguish Redactor's disinformation from real data (46.1% accuracy), suggesting the generated data is highly realistic.
Key Insights
- Input-Space Optimization: Previous "poisoning" attacks often optimize in the feature space of a pre-trained model. Redactor's authors show that in end-to-end training, feature spaces shift, making feature-based attacks ineffective. Redactor succeeds by optimizing directly in the input space.
- PDB as a Human Proxy: Using data programming to estimate a "conservative" decision boundary ensures the generated disinformation is actually labeled as a different class by the victim, even without seeing the victim's model.
- Data-Centric Privacy: This shifts the privacy conversation from model architecture to how individuals can protect themselves via the data they provide.
Limitations and Future Work
The defense relies on the assumption that generated disinformation will eventually be picked up by the same crawlers that took the original private data. Additionally, while CTGAN helps, some datasets require complex domain constraints that currently necessitate manual filtering. Future work may focus on more specialized image-generation components, as techniques used for tabular data do not translate directly to complex images.
Summary updated