Local Differential Privacy for Mixtures of Experts

Wissam Akretche, Frédéric LeBlanc, Mario MarchandOpenReview 2024 · Submission to NeurIPS 2024 · Rejected · OpenReview

Abstract

We introduce a new approach to the mixture of experts model that consists in imposing local differential privacy on the gating mechanism. This is theoretically justified by statistical learning theory. Notably, we provide generalization bounds specifically tailored for mixtures of experts, leveraging the one-out-of-$n$ gating mechanism rather than the more common $n$-out-of-$n$ mechanism. Moreover, through experiments, we show that our approach improves the generalization ability of mixtures of experts.
BibTeX
@misc{akretche2024local,
  author = {Wissam Akretche and Frédéric LeBlanc and Mario Marchand},
  title = {{Local Differential Privacy for Mixtures of Experts}},
  year = {2024},
  url = {https://openreview.net/forum?id=cdYIL6OQr6},
  howpublished = {OpenReview},
  note = {Submitted to NeurIPS 2024},
}
Figure