Introduction
Machine learning models trained on sensitive datasets, such as medical records or financial histories, are susceptible to Model Inversion (MI) attacks. In these scenarios, an attacker leverages partial information and model outputs to infer sensitive missing attributes. Researchers from IBM have identified that high feature influence—the degree to which a model relies on a specific attribute—is a primary driver of MI vulnerability. To address this, they propose "privacy-guided training," a set of techniques designed to minimize the influence of sensitive features without significantly compromising model utility.
Privacy-Guided Training Methods
The proposed framework introduces three modifications to the training of tree-based models (Decision Trees, Random Forests, and AdaBoost):
- Weight-based Penalty: This method assigns sensitivity weights to features. During split selection (e.g., using Gini impurity), sensitive features incur a penalty, encouraging the model to prioritize non-sensitive attributes unless the sensitive one offers a substantial information gain.
- Limiting Split Levels: By restricting how high in a tree a sensitive feature can appear, the model ensures these features only impact smaller, more specific data subsets, making them harder for an attacker to generalize and extract.
- Limiting Split Counts: This approach caps the total number of times a sensitive feature is used. A Breadth-First Search (BFS) strategy is employed to distribute these splits evenly across the tree structure, preventing localized high-influence nodes.
Experimental Evaluation
The methods were evaluated using the Nursery dataset (12,958 samples) and the GSS Marital Happiness dataset (24,455 samples). The researchers simulated both black-box and white-box attacks to measure the effectiveness of their countermeasures.
Key Results and Discussion
A central finding of the study is that privacy protection does not necessarily require a significant "utility tax." In many instances, reducing the importance of sensitive features had a negligible impact on overall model accuracy.
As shown in the results for the Nursery dataset, the researchers achieved a significant reduction in the importance of sensitive features while maintaining model accuracy around 92%. In the GSS dataset, reducing the "happiness" feature's importance from 0.315 to 0.01 caused white-box attack accuracy to drop from 80% to near-baseline levels (~60%). These results confirm that the success of inversion attacks is intrinsically linked to feature importance; by strategically driving down this importance, the risk of attribute leakage is mitigated.
Conclusion
Privacy-guided training offers a viable middle ground for deploying tree-based models on sensitive data. By focusing on reducing the influence of specific high-risk attributes rather than handicapping the entire model, researchers can protect individual privacy while maintaining high predictive performance. Future work will explore extending these influence-reduction strategies to neural network architectures and automating the identification of sensitive features.
Summary updated