Introduction
Graph Neural Networks (GNNs) have emerged as the state-of-the-art architecture for recommendation systems, leveraging the complex relational structures between users and items to predict preferences. However, in practice, these data points are often distributed across multiple organizations—such as different service providers or retailers—creating isolated data silos. Training a high-performing GNN typically requires a centralized global graph, which is often impossible due to strict data privacy regulations and the proprietary nature of user behavior data.
This paper introduces DP-FedRec, a novel federated learning (FL) framework designed to bridge these silos. By integrating Private Set Intersection (PSI), K-hop graph expansion, and Differential Privacy (DP), DP-FedRec enables collaborative GNN training while providing rigorous mathematical guarantees for user privacy.
The Challenges of Federated Graph Learning
Federated Learning traditionally allows multiple clients to train a model collaboratively without sharing raw data. However, applying FL to graph-structured data introduces two unique hurdles:
- The Non-IID Challenge: In standard FL, data is often assumed to be Independent and Identically Distributed (IID). In graph contexts, each client only observes a tiny, disconnected "subgraph." Because GNNs rely on neighborhood information to generate embeddings, these local silos lack the global context necessary to learn meaningful relationships, leading to poor model performance.
- Privacy Risks: While FL avoids raw data sharing, the exchange of model gradients or graph topology can still leak sensitive information. Adversaries can potentially reverse-engineer gradients to reveal private user interactions or infer the existence of specific edges (relationships) within the graph.
The DP-FedRec Framework
To address these issues, the authors propose a four-stage pipeline that balances the need for global context with the requirement for local privacy.
1. Private Set Intersection (PSI)
Before training begins, clients must identify overlapping users to facilitate knowledge sharing. DP-FedRec employs a cryptographic PSI protocol, allowing clients to discover common user IDs without revealing any information about users they do not share.
2. K-hop Graph Expansion
To mitigate the non-IID problem, the framework allows clients to "expand" their local subgraphs. If Client A and Client B share a user, Client A can request information regarding that user’s neighbors (up to $K$ hops away) from Client B. This process effectively "borrows" context from other silos, providing the local GNN with a much richer structural environment.
3. Differential Privacy (DP) for Topology and Weights
To ensure that the expansion process does not compromise privacy, the authors apply Laplacian noise to two distinct layers of the graph data:
- Topology Protection: The LAPGRAPH algorithm is used to perturb the adjacency matrix, masking the exact connections between users and items.
- Weight Protection: Noise is added directly to edge weights, such as specific movie ratings or purchase frequencies, ensuring that individual interactions remain private.
4. Federated Training
With the expanded and noisy subgraphs prepared, clients train local Graph Convolutional Networks (GCNs). The resulting model parameters are then aggregated via a central server using standard federated averaging techniques to produce a global model.
Experimental Evaluation
The framework was evaluated using two benchmark recommendation datasets:
- Epinions: A consumer review dataset partitioned into 12 clients based on item categories.
- MovieLens 1M: A classic movie rating dataset partitioned into 8 clients.
Key Results
- Accuracy Improvements: On the Epinions dataset, DP-FedRec with $K=10$ expansion achieved a Mean Absolute Error (MAE) of 0.8585. This significantly outperformed the standard federated baseline (FedGraphNN), which yielded an MAE of 0.8674.
- The Privacy-Utility Trade-off: Interestingly, the benefits of K-hop expansion often outweighed the accuracy loss typically associated with Differential Privacy. In several tests, DP-FedRec performed better than non-private baselines that lacked graph expansion, suggesting that solving the non-IID problem is more critical for GNN performance than maintaining perfect data purity.
- Computational Overhead: The "noising" process proved efficient, taking between 3 and 517 seconds depending on dataset size. This indicates that the privacy-preserving steps are manageable within the context of offline recommendation model training.
Conclusion and Future Directions
DP-FedRec demonstrates that it is possible to train high-quality recommendation models across data silos without sacrificing privacy. By allowing clients to privately share neighborhood context, the framework overcomes the structural limitations of local subgraphs.
However, limitations remain regarding scalability. The communication costs currently scale with the square of the number of clients ($l^2$) and the square of the number of nodes ($|V|^2$), which may pose challenges for massive, hyper-distributed networks. Future research may explore "universal DP" mechanisms to handle weights and topology simultaneously, as well as optimization techniques to reduce the communication footprint for large-scale deployment.
Summary updated