Mixed-Privacy Forgetting in Deep Networks

Aditya Golatkar, Alessandro Achille, Avinash Ravichandran, Marzia Polito, Stefano Soatto

Abstract: We show that the influence of a subset of the training samples can be removed -- or "forgotten" -- from the weights of a network trained on large-scale image classification tasks, and we provide strong computable bounds on the amount of remaining information after forgetting. Inspired by real-world applications of forgetting techniques, we introduce a novel notion of forgetting in mixed-privacy setting, where we know that a "core" subset of the training samples does not need to be forgotten. While this variation of the problem is conceptually simple, we show that working in this setting significantly improves the accuracy and guarantees of forgetting methods applied to vision classification tasks. Moreover, our method allows efficient removal of all information contained in non-core data by simply setting to zero a subset of the weights with minimal loss in performance. We achieve these results by replacing a standard deep network with a suitable linear approximation. With opportune changes to the network architecture and training procedure, we show that such linear approximation achieves comparable performance to the original network and that the forgetting problem becomes quadratic and can be solved efficiently even for large models. Unlike previous forgetting methods on deep networks, ours can achieve close to the state-of-the-art accuracy on large scale vision tasks. In particular, we show that our method allows forgetting without having to trade off the model accuracy.

Introduction

With regulations like GDPR granting users the "right to be forgotten," AI providers must be able to delete a user's data from their trained models. Simply deleting the data from a database is insufficient; the data's influence remains embedded in the model's weights. The standard solution—retraining the model from scratch—is computationally prohibitive for large-scale deep networks.

This paper, presented at CVPR 2021, addresses the need for "machine unlearning" by introducing Mixed-Linear Forgetting (ML-Forgetting). This method allows deep networks to "forget" specific data cohorts with strong privacy guarantees and minimal loss in accuracy.

The Mixed-Privacy Framework

The authors propose a "mixed-privacy" setting, assuming training data consists of a stable core dataset $D_c$ (like ImageNet) that will never need to be forgotten, and a user dataset $D$ that might. The process involves three main steps:

  1. Model Splitting & Linearization: A standard non-linear network is trained on the core data. To incorporate user data, the authors use a first-order Taylor expansion rather than full fine-tuning. The final model output is the core model's output plus a linear perturbation: $f(x) \approx f_{w_c}(x) + \nabla_w f_{w_c}(x) \cdot w_u$.
  2. Quadratic Optimization: By training the user weights $w_u$ using a Mean Squared Error (MSE) loss, the optimization problem becomes strongly convex. This transformation allows for efficient, closed-form "scrubbing" updates when a forgetting request is received.
  3. Architectural Refinement: To ensure the linearized model matches the performance of a full DNN, standard ReLUs are replaced with Leaky ReLUs, which improves gradient flow in linearized approximations.
Figure 1: Test error comparison across multiple datasets showing that the Mixed-Linear approach achieves accuracy comparable to standard DNNs, significantly outperforming simple last-layer feature training.

Key Results and Insights

The researchers evaluated ML-Forgetting on several benchmarks, including Caltech-256, MIT-67, and Stanford Dogs. The results highlight several advantages over traditional methods:

  • Accuracy Parity: As shown in Figure 1, ML-Forgetting achieves test accuracy comparable to standard non-linear DNNs. This demonstrates that linearization is a practical tool for real-world vision tasks, not just a theoretical one.
  • Efficiency: The forgetting procedure is 16x to 25x faster than retraining from scratch. Because the problem is convex, it can be solved with a few passes of SGD.
  • Privacy Effectiveness: Using "readout functions" (attacks designed to see if a model remembers data), the authors showed that an ML-Forgotten model is statistically indistinguishable from a model that was never shown the data.
Figure 2: Performance metrics comparing the proposed method to baselines. The proposed method maintains low error on retained/test sets while effectively "forgetting" (matching the retrained baseline) with significantly lower retrain time.

Conclusion

ML-Forgetting represents a significant step forward in machine unlearning. By acknowledging that not all data is subject to deletion, the authors create a "safe harbor" for complex non-linear features in the core weights, leaving only the final adaptations in a form that is easy to scrub. While the method currently relies on $L_2$ regularization and high-quality core data, it provides a robust framework for forgetting without the traditional trade-off in model accuracy.

Summary updated

Abstract

We show that the influence of a subset of the training samples can be removed -- or "forgotten" -- from the weights of a network trained on large-scale image classification tasks, and we provide strong computable bounds on the amount of remaining information after forgetting. Inspired by real-world applications of forgetting techniques, we introduce a novel notion of forgetting in mixed-privacy setting, where we know that a "core" subset of the training samples does not need to be forgotten. While this variation of the problem is conceptually simple, we show that working in this setting significantly improves the accuracy and guarantees of forgetting methods applied to vision classification tasks. Moreover, our method allows efficient removal of all information contained in non-core data by simply setting to zero a subset of the weights with minimal loss in performance. We achieve these results by replacing a standard deep network with a suitable linear approximation. With opportune changes to the network architecture and training procedure, we show that such linear approximation achieves comparable performance to the original network and that the forgetting problem becomes quadratic and can be solved efficiently even for large models. Unlike previous forgetting methods on deep networks, ours can achieve close to the state-of-the-art accuracy on large scale vision tasks. In particular, we show that our method allows forgetting without having to trade off the model accuracy.
BibTeX
@inproceedings{golatkar2021mixed,
  author = {Aditya Golatkar and Alessandro Achille and Avinash Ravichandran and Marzia Polito and Stefano Soatto},
  title = {{Mixed-Privacy Forgetting in Deep Networks}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2021},
  url = {https://openreview.net/forum?id=zDEmeN0erO},
}
Figure