Interpretable and Differentially Private Predictions

Frederik Harder, Matthias Bauer, Mijung Park

Modern machine learning faces a critical challenge: the "triangular" trade-off between accuracy, privacy, and interpretability. While deep learning models achieve state-of-the-art accuracy, they often operate as "black boxes" that can inadvertently leak sensitive training data. This paper, presented at AAAI-20, introduces a novel architecture designed to navigate these competing demands.

The triangular trade-off between accuracy, privacy, and interpretability.

The Challenge of Modern AI Constraints

In the regulatory landscape defined by the General Data Protection Regulation (GDPR), machine learning practitioners must address two major hurdles: the "Right to Explanation" and strict data privacy. Users increasingly demand to know why an algorithm reached a specific decision, yet models must also ensure that no individual's data can be reconstructed from the model's parameters.

Standard deep neural networks are notoriously difficult to interpret. Post-hoc methods like saliency maps often become noisy and nonsensical when the model is trained with Differential Privacy (DP). Conversely, inherently interpretable models like logistic regression often lack the capacity to achieve high accuracy on complex, high-dimensional datasets. This research seeks a "Goldilocks" model that satisfies all three requirements simultaneously.

Methodology: Locally Linear Maps (LLM)

The authors propose Locally Linear Maps (LLM), a family of models that approximate complex decision boundaries using a collection of simple linear functions. Instead of a single monolithic neural network, the LLM uses $M$ linear maps for each class.

Architecture and Training

For any given input, the model employs a "softmax" weighting mechanism to determine which linear maps are most relevant. This makes the model piece-wise linear: locally, it behaves like a simple, understandable linear classifier, while globally, it can capture the complex patterns necessary for high accuracy.

To ensure privacy, the model is trained using DP-SGD (Stochastic Gradient Descent), which involves clipping gradients and adding Gaussian noise. To combat the "curse of dimensionality"—where high-dimensional data requires more noise, thus destroying accuracy—the authors utilize the Johnson-Lindenstrauss transform. By projecting high-dimensional inputs into a lower-dimensional space using a fixed, data-independent matrix, they improve the privacy-accuracy trade-off without consuming the privacy budget.

Key Insights and Results

The LLM was evaluated on MNIST, Fashion-MNIST, and a real-world medical dataset from Henan Renmin Hospital. The results demonstrate that LLMs can achieve competitive accuracy while maintaining superior interpretability compared to traditional private models.

  • Accuracy: On MNIST, the LLM achieved 94.2% accuracy with a privacy budget of $\epsilon=2$. On the medical dataset, it reached 82.0% accuracy, trailing a non-private deep neural network by only 2%.
  • Inherent Interpretability: Unlike post-hoc explanations for black-box models, LLMs are inherently interpretable. The linear maps act as "prototypes" or templates. While standard attribution maps degrade into noise under high privacy constraints, LLM filters remain recognizable.
An LLM prototype for a clothing item (Fashion-MNIST), showing how the model maintains interpretability even under privacy constraints.

Conclusion

Locally Linear Maps provide a robust middle ground for privacy-sensitive applications. By combining the simplicity of linear models with the flexibility of piece-wise approximation and the efficiency of random projections, the authors demonstrate that it is possible to build models that are accurate, private, and explainable. Future work will focus on scaling these models to higher-resolution datasets and exploring more flexible DP-compliant gradient information.

Summary updated

Abstract

Interpretable predictions, which clarify why a machine learning model makes a particular decision, can compromise privacy by revealing the characteristics of individual data points. This raises the central question addressed in this paper: Can models be interpretable without compromising privacy? For complex “big” data fit by correspondingly rich models, balancing privacy and explainability is particularly challenging, such that this question has remained largely unexplored. In this paper, we propose a family of simple models with the aim of approximating complex models using several locally linear maps per class to provide high classification accuracy, as well as differentially private explanations on the classification. We illustrate the usefulness of our approach on several image benchmark datasets as well as a medical dataset.
BibTeX
@inproceedings{harder2020interpretable,
  author = {Frederik Harder and Matthias Bauer and Mijung Park},
  title = {{Interpretable and Differentially Private Predictions}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i04.5827},
}
Figure