TL;DR
This paper introduces Protected User Consent (PUC), a framework that prevents machine learning models from penalizing users who choose not to share optional personal data. By using a novel data augmentation technique called PUCIDA, the authors ensure that a user's decision to "prefer not to say" doesn't inadvertently become a predictive signal that leads to discriminatory outcomes.
What Problem Does This Paper Address?
In many modern automated systems—like health insurance pricing—users are asked for mandatory "base" features (e.g., age, residence) but can voluntarily provide "optional" features (e.g., data from a fitness tracker) in exchange for potential benefits.
The core problem is Availability Inference. Standard machine learning models are "greedy"; they learn from every available signal, including the absence of data. If fit individuals are more likely to share their fitness data and less fit individuals are not, a model will learn that "missing data = unfit." Consequently, a user who exercises their legal right to privacy (under regulations like GDPR or CCPA) is effectively penalized with a higher insurance premium just for staying silent. The authors argue that the decision not to share data is private information in itself and should not be used to a user's disadvantage.
Methods
The authors formalize a requirement called Availability Inference Restriction (AIR), which mandates that a model's prediction for a non-sharer must not be worse than a "base model" trained only on mandatory features. To achieve this, they propose:
- Protected User Consent (PUC): A theoretical optimal predictor that satisfies AIR. It provides the best possible estimate for sharers using all data, while for non-sharers, it reverts to the best estimate based strictly on base features, marginalizing out the missingness signal.
- PUCIDA (PUC-inducing Data Augmentation): A model-agnostic training strategy. Instead of just training on the raw data, PUCIDA creates synthetic samples. For every instance where a user did share optional data, a synthetic twin is created where that data is hidden (set to N/A), but the label remains the same.
- Weighting Scheme: To ensure the model doesn't just learn a new bias, PUCIDA uses a specific weighting and sampling strategy to make the label distribution for "missing" data identical to the overall label distribution. This effectively "blinds" the model to the predictive power of the missingness itself.
Datasets
The authors evaluated their approach on eight real-world datasets and two synthetic benchmarks:
- Classification: Diabetes (diab), COMPAS (recidivism), UCI Adult (income), Water treatment (water), and Horse colic (colic).
- Regression: California Housing (calif), ACSIncome (income), and a Health Insurance dataset (insurance).
- Synthetic Benchmarks: Used to verify theoretical convergence rates and the model-agnostic nature of the approach.
Key Insights and Ideas
- Privacy vs. Performance is not a Zero-Sum Game: The authors prove that a decision-maker can still benefit from the extra data provided by sharers (improving overall accuracy) without systematically disadvantaging those who choose not to share.
- "Fairness through Unawareness" Fails: Simply dropping the optional feature for non-sharers isn't enough. If the fact that a feature is missing is correlated with the outcome, complex models (like Neural Networks or Gradient Boosted Trees) will find a way to exploit that correlation.
- Model Agnosticism: Because PUCIDA is a data-level intervention, it works across various architectures, from simple Decision Trees to Multi-layer Perceptrons.
Key Results
- Quantifying the Penalty: Without protection, non-sharers faced significant disadvantages. In the
incomeandcompasdatasets, full-feature models assigned outcomes that were up to 18% less favorable for non-sharers compared to a base model. - Eliminating the Gap: Applying PUCIDA virtually eliminated this "fairness gap" across all tested model types (Random Forests, MLPs, etc.), bringing the PUC-Gap close to zero.
- Convergence: The authors proved and empirically verified that the error of a PUC-compliant model trained with PUCIDA converges to the theoretical optimum at a rate of $O(1/N)$.
Limitations and Future Work
- Strategic Behavior: The current work assumes users are either purely strategic (sharing only if it helps them) or purely privacy-motivated. Real-world behavior is often a messy mix of both, which could be modeled in future work.
- Long-term Dynamics: The paper focuses on a static snapshot. Future research could explore how PUC-compliant models affect user behavior over time—for instance, does it encourage more people to share data if they know they won't be penalized for staying silent?
- Computational Overhead: While model-agnostic, the "exhaustive augmentation" variant of PUCIDA can increase dataset size, which might be a concern for extremely large-scale applications.
Summary updated