Differentially private secure multi-party computation for federated learning in financial applications

David Byrd, Antigoni Polychroniadou

TL;DR

This paper presents a robust framework for financial institutions to collaboratively train machine learning models without ever sharing sensitive client data. By combining Federated Learning (FL) with both Differential Privacy (DP) and Secure Multi-Party Computation (MPC), the authors create a system that protects against data leakage from both the central server and colluding malicious participants, all while maintaining high accuracy on real-world fraud detection tasks.

What Problem Does This Paper Address?

Financial institutions sit on goldmines of data, but regulatory hurdles and privacy concerns often prevent them from pooling this data to train better models (e.g., for fraud detection or credit scoring).

While Federated Learning allows models to be trained locally on separate servers, it isn't a silver bullet. Research has shown that a curious "honest-but-curious" server can reverse-engineer private data just by looking at the model weights sent by clients. Adding Differential Privacy (noise) can stop this, but too much noise makes the model useless. Secure Multi-Party Computation can hide individual weights from the server, but if several clients collude, they can still "out" a single honest participant. This paper addresses the need for a "defense-in-depth" approach that balances these trade-offs specifically for the high-stakes financial sector.

Methods

The authors propose a protocol called $\Pi_{PPFL}$, which integrates three key layers:

  1. Federated Logistic Regression: Clients train local models and only share the weights ($w$).
  2. Secure Multi-Party Computation (MPC): Based on the Bonawitz protocol, clients pair up and share secret random offsets. One client adds the offset to their weights, and the other subtracts it. When the server averages all weights, the offsets cancel out perfectly, but the server never sees any individual client's true weights.
  3. Differential Privacy (DP): Before the MPC step, clients add Laplacian noise to their weights. This ensures that even if $n-1$ clients collude to reveal the last client's masked weights, they only get a "noisy" version that preserves the individual's privacy.

A standout feature of this research is the use of ABIDES, an agent-based discrete event simulator. Unlike many papers that ignore network realities, this study simulates nanosecond-level timing, message priority queues, and network "jitter" to provide realistic performance estimates.

Datasets

The authors evaluated their protocol using the Kaggle Credit Card Fraud (2013) dataset.

  • Characteristics: 284,807 records with 30 features.
  • Challenge: It is extremely class-imbalanced; only 492 transactions (less than 0.2%) are labeled as fraudulent. This makes it a "needle in a haystack" problem where standard accuracy metrics are misleading.

Key Insights and Ideas

  • The Power of Crowds: The authors found that as the number of participating clients increases, the "privacy budget" improves. With more participants, you can add less noise per person to achieve the same level of global privacy, leading to much more accurate models.
  • Optimizing the "Neighborhood": In a standard MPC setup, every client talks to every other client ($O(N^2)$ complexity). The authors demonstrated that clients only need to exchange keys with a small "neighborhood" of size $\log(N)$ to maintain security while significantly speeding up the protocol.
  • Metric Matters: Because of the fraud dataset's imbalance, the authors used the Matthews Correlation Coefficient (MCC). They showed that a naive model (predicting "no fraud" every time) gets 99.8% accuracy but an MCC of zero, whereas their private federated model achieves high MCC scores, proving it actually learns the minority class.

Key Results

  • Accuracy: For populations of 200+ clients, the model maintained high accuracy (comparable to "in the clear" training) even with a privacy loss parameter ($\epsilon$) as low as $5e^{-4}$.
  • Scalability: The simulation showed that a 500-party federation could complete a training iteration in roughly 17 seconds, including encryption and communication overhead.
  • Collusion Resistance: The paper mathematically demonstrates that even if all but one client collude, the "noisy" weights recovered would still contain enough uncertainty to protect the honest party's specific data points.

Limitations and Future Work

  • Semi-Honest Adversaries: The current protocol assumes "semi-honest" participants (they follow the rules but try to learn extra info). It does not yet account for "active" or "malicious" participants who might send fake weights to poison the global model.
  • Model Complexity: The study focused on Logistic Regression. While effective for fraud, future work is needed to see how this dual DP-MPC approach scales to deep neural networks with millions of parameters.
  • Communication Overhead: While $\log(N)$ neighborhoods help, the initial setup (Diffie-Hellman key exchange) still represents a bottleneck for very large-scale deployments.

Summary updated

Abstract

Federated Learning enables a population of clients, working with a trusted server, to collaboratively learn a shared machine learning model while keeping each client's data within its own local systems. This reduces the risk of exposing sensitive data, but it is still possible to reverse engineer information about a client's private data set from communicated model parameters. Most federated learning systems therefore use differential privacy to introduce noise to the parameters. This adds uncertainty to any attempt to reveal private client data, but also reduces the accuracy of the shared model, limiting the useful scale of privacy-preserving noise. A system can further reduce the coordinating server's ability to recover private client information, without additional accuracy loss, by also including secure multiparty computation. An approach combining both techniques is especially relevant to financial firms as it allows new possibilities for collaborative learning without exposing sensitive client data. This could produce more accurate models for important tasks like optimal trade execution, credit origination, or fraud detection. The key contributions of this paper are: We present a privacy-preserving federated learning protocol to a non-specialist audience, demonstrate it using logistic regression on a real-world credit card fraud data set, and evaluate it using an open-source simulation platform which we have adapted for the development of federated learning systems.
BibTeX
@inproceedings{byrd2020differentially,
  author = {David Byrd and Antigoni Polychroniadou},
  title = {{Differentially private secure multi-party computation for federated learning in financial applications}},
  booktitle = {ICAIF},
  year = {2020},
  url = {https://openreview.net/forum?id=KPgt0jXf6af},
}

Loading related content...

Figure