PICSR: Prototype-Informed Cross-Silo Router for Federated Learning (Student Abstract)

Eric Enouen, Sebastian Caldas, Mononito Goswami, Artur Dubrawski

Introduction

Federated Learning (FL) enables multiple institutions, such as hospitals or financial entities, to collaboratively train machine learning models without sharing sensitive raw data. While this approach preserves privacy, it often struggles with data heterogeneity—the inherent differences in data distributions across various "silos." Most existing FL methods aim for a single global model, which can become a "black box" that obscures institutional differences and lacks interpretability.

The Prototype-Informed Cross-Silo Router (PICSR) addresses these challenges by employing a "mixture of experts" (MoE) architecture. By grounding its routing decisions in representative data "prototypes," PICSR not only enhances predictive performance but also provides a transparent map of how different institutions' data relates to one another.

Methodology: Prototype-Informed Routing

PICSR treats each participating institution as an "expert" and utilizes a specialized router to dynamically weight their contributions for any given input. The core innovation lies in making this routing process interpretable through data grounding.

The PICSR architecture involves three main steps: computing distances to prototypes, inverting these to find similarity, and weighting expert predictions.

The process follows three distinct stages:

  1. Embedding and Prototype Comparison: An input sample $x$ is mapped into an embedding space $f(x)$. The system then calculates the $L_2$ distance between this embedding and the prototypes—the average feature vectors—of each silo.
  2. Similarity Computation: These distances are inverted to determine similarity. A sample that is closer to a specific silo's prototype is considered more similar to that institution's typical data.
  3. Weighted Prediction: The router assigns weights based on these similarities. The final output is a weighted sum of the predictions from all expert models ($E_1(x), \dots, E_K(x)$).

Experimental Setup

The researchers evaluated PICSR using the Fed-Heart-Disease (FHD) dataset from the FLamby benchmark. This dataset comprises 740 samples across four distinct geographic locations: Cleveland, Hungary, Switzerland, and VA Long Beach. The task involves predicting the presence of heart disease based on 13 clinical features.

Key Insights: Visualizing Collaboration and Outliers

A significant advantage of PICSR is its ability to visualize institutional collaboration. By inspecting the router's weights, stakeholders can understand which models are most influential for specific data distributions.

The routing distribution across test datasets reveals how PICSR leverages different expert models based on data similarity.

Analysis of the routing distribution yielded several key insights:

  • Inter-Silo Collaboration: Cleveland and Hungary silos exhibit similar patient distributions, leading the router to frequently utilize both models for samples from either location.
  • Identifying Outliers: The Switzerland silo was identified as a significant outlier. Due to its unique patient population, the router predominantly assigned weight back to its own local expert model (approx. 95% of the time), rather than relying on others.
  • Explicit Contributions: Unlike standard Federated Averaging (FedAvg), PICSR maintains the identity of local experts, making the contribution of each partner measurable and explicit.

Performance Results

PICSR demonstrates that interpretability does not come at the cost of accuracy. The framework matched or exceeded standard baselines like FedAvg and FedProx across the test silos.

  • Hungary Silo: PICSR achieved 80.67% accuracy, surpassing the next best federated method (79.78%).
  • VA Silo: The model reached 82.67% accuracy, a substantial improvement over the local-only baseline of 76.89%.
  • Robustness: The system remained stable even for small datasets, such as the Switzerland silo ($n=46$), by correctly identifying when to rely on local expertise.

Conclusion and Future Work

PICSR provides a promising path toward more transparent and effective federated learning in high-stakes domains. While currently a proof-of-concept, future research will focus on scaling the prototype-informed routing to larger, high-dimensional datasets and implementing additional privacy safeguards, such as Differential Privacy, for the shared prototypes.

Summary updated

Abstract

Federated Learning is an effective approach for learning from data distributed across multiple institutions. While most existing studies are aimed at improving predictive accuracy of models, little work has been done to explain knowledge differences between institutions and the benefits of collaboration. Understanding these differences is critical in cross-silo federated learning domains, e.g., in healthcare or banking, where each institution or silo has a different underlying distribution and stakeholders want to understand how their institution compares to their partners. We introduce Prototype-Informed Cross-Silo Router (PICSR) which utilizes a mixture of experts approach to combine local models derived from multiple silos. Furthermore, by computing data similarity to prototypical samples from each silo, we are able to ground the router’s predictions in the underlying dataset distributions. Experiments on a real-world heart disease prediction dataset show that PICSR retains high performance while enabling further explanations on the differences among institutions compared to a single black-box model.
BibTeX
@inproceedings{enouen2024picsr,
  author = {Eric Enouen and Sebastian Caldas and Mononito Goswami and Artur Dubrawski},
  title = {{PICSR: Prototype-Informed Cross-Silo Router for Federated Learning (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30438},
}
Figure