Introduction
Federated Learning (FL) enables multiple institutions, such as hospitals or financial entities, to collaboratively train machine learning models without sharing sensitive raw data. While this approach preserves privacy, it often struggles with data heterogeneity—the inherent differences in data distributions across various "silos." Most existing FL methods aim for a single global model, which can become a "black box" that obscures institutional differences and lacks interpretability.
The Prototype-Informed Cross-Silo Router (PICSR) addresses these challenges by employing a "mixture of experts" (MoE) architecture. By grounding its routing decisions in representative data "prototypes," PICSR not only enhances predictive performance but also provides a transparent map of how different institutions' data relates to one another.
Methodology: Prototype-Informed Routing
PICSR treats each participating institution as an "expert" and utilizes a specialized router to dynamically weight their contributions for any given input. The core innovation lies in making this routing process interpretable through data grounding.
The process follows three distinct stages:
- Embedding and Prototype Comparison: An input sample $x$ is mapped into an embedding space $f(x)$. The system then calculates the $L_2$ distance between this embedding and the prototypes—the average feature vectors—of each silo.
- Similarity Computation: These distances are inverted to determine similarity. A sample that is closer to a specific silo's prototype is considered more similar to that institution's typical data.
- Weighted Prediction: The router assigns weights based on these similarities. The final output is a weighted sum of the predictions from all expert models ($E_1(x), \dots, E_K(x)$).
Experimental Setup
The researchers evaluated PICSR using the Fed-Heart-Disease (FHD) dataset from the FLamby benchmark. This dataset comprises 740 samples across four distinct geographic locations: Cleveland, Hungary, Switzerland, and VA Long Beach. The task involves predicting the presence of heart disease based on 13 clinical features.
Key Insights: Visualizing Collaboration and Outliers
A significant advantage of PICSR is its ability to visualize institutional collaboration. By inspecting the router's weights, stakeholders can understand which models are most influential for specific data distributions.
Analysis of the routing distribution yielded several key insights:
- Inter-Silo Collaboration: Cleveland and Hungary silos exhibit similar patient distributions, leading the router to frequently utilize both models for samples from either location.
- Identifying Outliers: The Switzerland silo was identified as a significant outlier. Due to its unique patient population, the router predominantly assigned weight back to its own local expert model (approx. 95% of the time), rather than relying on others.
- Explicit Contributions: Unlike standard Federated Averaging (FedAvg), PICSR maintains the identity of local experts, making the contribution of each partner measurable and explicit.
Performance Results
PICSR demonstrates that interpretability does not come at the cost of accuracy. The framework matched or exceeded standard baselines like FedAvg and FedProx across the test silos.
- Hungary Silo: PICSR achieved 80.67% accuracy, surpassing the next best federated method (79.78%).
- VA Silo: The model reached 82.67% accuracy, a substantial improvement over the local-only baseline of 76.89%.
- Robustness: The system remained stable even for small datasets, such as the Switzerland silo ($n=46$), by correctly identifying when to rely on local expertise.
Conclusion and Future Work
PICSR provides a promising path toward more transparent and effective federated learning in high-stakes domains. While currently a proof-of-concept, future research will focus on scaling the prototype-informed routing to larger, high-dimensional datasets and implementing additional privacy safeguards, such as Differential Privacy, for the shared prototypes.
Summary updated