Preventing Adversarial Use of Datasets through Fair Core-Set Construction

Benjamin Spector, Ravi Kumar, Andrew Tomkins

TL;DR

Instead of using Differential Privacy or Federated Learning, this paper proposes a "core-set" approach to privacy: publishing only a strategically selected subset of a dataset. By carefully choosing which instances to include, researchers can maintain high performance on a primary target task while intentionally sabotaging performance on unwanted or "adversarial" tasks.

What Problem Does This Paper Address?

The modern machine learning pipeline is built on a paradox: state-of-the-art models require massive datasets, yet users are increasingly (and rightfully) protective of their personal information. While we might want our data to improve search results or medical diagnoses, we fear it being used for malicious profiling or extracting sensitive secondary attributes.

Existing solutions like Differential Privacy (adding noise) or Federated Learning (keeping data local) are powerful but can be complex to implement or damaging to model utility. This paper explores a simpler, more structural alternative: data curation as a privacy tool. The goal is to release a version of a dataset that is "honest" about the primary task but "deceptive" or "uninformative" about sensitive secondary tasks.

Methods

The authors propose a two-step framework for constructing these privacy-preserving core-sets:

  1. Approximation: Train one or more models on the entire dataset to establish a "ground truth" for the target function within a specific model family.
  2. Optimization: Select a subset (core-set) of size $k$ that minimizes a custom loss function. This loss function balances two competing goals:
  3. Fidelity: The subset should align closely with the model trained on the full data for the primary task.
  4. Obfuscation: The subset should make it difficult to recover a specific "secret" attribute.

For linear models, they use a loss function that combines squared error for the primary task with a penalty term for the secret task. They suggest two obfuscation strategies: "hiding" the secret by choosing samples where the attribute is uniform, or "planting" a synthetic secret to actively mislead a learner.

For neural networks, the authors use average gradient magnitude during training as a proxy for sample difficulty. The intuition is that "easy" examples (low gradient norm) generalize well for small core-sets, while "hard" or unrepresentative examples do not. By selecting samples based on the ratio of their gradient norms across different tasks, they can favor one task over another.

Datasets

The paper evaluates the approach on two fronts:

  • Synthetic Linear Regression: A generated dataset with three features ($x_1, x_2, x_3$), a primary label ($y$), and a secret label ($z$).
  • CIFAR-100: A standard image dataset that features a natural hierarchy. The authors treat coarse-grained classification (20 super-classes) as the primary task and fine-grained classification (100 classes) as the unwanted task.

Key Insights and Ideas

  • Non-Representative is Good: Traditionally, we want subsets to be representative of the whole. Here, the "aha moment" is that strategic unrepresentativeness is a feature, not a bug. By picking "weird" samples that still support the main task, you can break the correlations an adversary would need for other tasks.
  • Gradient Norms as a Privacy Lever: In deep learning, not all data points are created equal. The discovery that you can mask a sub-task by filtering for specific gradient norm ratios (as shown in Figure 2) provides a practical way to apply core-set theory to complex, non-linear models.
  • Simplicity of Implementation: Unlike methods that require modifying training loops or adding complex noise layers, this approach only requires deletion. Once the core-set is selected, it can be used with any standard training algorithm.
Figure 2: CNN CIFAR-100 test accuracy trained on k = 5000 class-balanced core-sets versus the average gradient norm of the examples used. Core-sets made of smaller-normed examples do better.

Key Results

The results demonstrate that core-sets can effectively "decouple" related tasks:

  • Linear Regression: As shown in Figure 3, the core-set actually improved the $r^2$ for the primary task $y$ (from 0.89 to 0.99) while turning the $r^2$ for the secret task $z$ negative (-0.35). A negative $r^2$ means the core-set actively misleads a linear learner, performing worse than a simple mean-based guess.
  • CIFAR-100: The "fine-grained-masking" core-set maintained a respectable 39.55% accuracy on coarse labels but dropped fine-grained accuracy to 16.40%—significantly lower than the 22.83% achieved by a purely random subset (see Figure 4).
Figure 1: Visualization of the datasets (green/red) and core-sets (blue) for each feature and label from Table 1.

Limitations and Future Work

The authors acknowledge several hurdles:

  • Adversarial Priors: If an attacker already has a very strong prior understanding of the feature distributions, they might be able to detect the "bias" in the core-set and correct for it.
  • Dataset Size: For deep learning, the method works best when the original dataset is much larger than the required core-set. If the dataset is small, there aren't enough "unrepresentative" samples to choose from to effectively mask secondary tasks.
  • Task Similarity: Hiding fine-grained labels while keeping coarse labels is particularly difficult because the tasks are so closely related. The authors suggest testing these ideas on even larger datasets like ImageNet in the future.

Summary updated

Abstract

We propose improving the privacy properties of a dataset by publishing only a strategically chosen "core-set" of the data containing a subset of the instances. The core-set allows strong performance on primary tasks, but forces poor performance on unwanted tasks. We give methods for both linear models and neural networks and demonstrate their efficacy on data.
BibTeX
@misc{spector2019preventing,
  author = {Benjamin Spector and Ravi Kumar and Andrew Tomkins},
  title = {{Preventing Adversarial Use of Datasets through Fair Core-Set Construction}},
  year = {2019},
  eprint = {1910.10871},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1910.10871},
}
Figure