Measuring Participant Contributions in Decentralized Federated Learning

Honoka Anada, Tatsuya Kaneko, Shinya Takamaeda-YamazakiarXiv 2025 · arXiv · OpenReview

TL;DR

This paper introduces TRIP-Shapley, a novel method for evaluating individual client contributions in decentralized federated learning (DFL). It addresses the challenges of limited client visibility and the indirect propagation of contributions in DFL, offering a robust and scalable solution that can even handle dishonest participants.

What Problem Does This Paper Address?

Federated Learning (FL) allows multiple clients to train a shared model without revealing their private data. Decentralized FL (DFL) takes this a step further by removing the central server, allowing clients to communicate directly. While evaluating client contributions is vital for incentivizing participation and ensuring fairness, existing methods are designed for centralized FL. These methods struggle with DFL due to two main issues: 1) clients can only see models from their immediate neighbors, not the entire network, and 2) a client's contribution isn't a one-off event but rather a gradual influence that propagates through peer-to-peer exchanges over time. This makes it difficult to accurately attribute the final model's performance to individual clients.

Methods

The core of TRIP-Shapley is its ability to trace the propagation of contributions over multiple rounds. Instead of simply summing up contributions from each round (which would be inaccurate in DFL), TRIP-Shapley models how a client's update in one round can indirectly influence others in subsequent rounds.

Key methodological components include:

  • Local Contribution Vectors (LCVs): Each client calculates LCVs, which represent its local assessment of contributions from itself and its neighbors. This is done by evaluating the model's performance with different subsets of clients, but crucially, it uses pre-training models to isolate the current round's contribution.
  • Coordinator-based Aggregation: A lightweight coordinator node collects these LCVs. It then uses a recursive update rule to estimate the overall contribution of each client by tracing the influence propagation across rounds. This avoids the need for the coordinator to handle actual model parameters.
  • Dishonesty Countermeasures: The paper addresses two types of dishonest behavior:
  • Pre-training Model Falsification (D1): Clients might send fake pre-training models to inflate their LCVs. The countermeasure (C1) involves clients filtering out suspiciously low-accuracy pre-training models.
  • LCV Falsification (D2): Clients might directly send manipulated LCVs to the coordinator. The countermeasure (C2) uses outlier detection on reported LCVs to identify and correct manipulated values.

Datasets

The paper uses standard image classification datasets for evaluation:

  • CIFAR-10
  • FashionMNIST (FMNIST)

These datasets are used to train lightweight convolutional neural networks (CNNs). The experiments explore various data distributions, including:

  • IID (Independent and Identically Distributed): Data is evenly distributed among clients.
  • Non-IID: Data has different label distributions across clients.
  • Sizes: Data is IID but varies in quantity per client.
  • Noisy-images: Images have varying levels of Gaussian noise.
  • Noisy-labels: Datasets contain a proportion of incorrectly labeled samples.

Key Insights and Ideas

  • The "Inter-round Dependency" is Crucial: The most significant insight is that simply summing round-wise contributions, a common practice in centralized FL, is fundamentally flawed for DFL. The indirect influence of a client's early contribution can manifest much later, making a temporal tracing mechanism essential.
  • Local Observability is Sufficient: TRIP-Shapley cleverly leverages local information (LCVs) to estimate global contributions. This is a key enabler for DFL, as clients don't need to see everyone's models.
  • Robustness to Dishonesty is Achievable: The proposed countermeasures (pre-training model filtering and LCV outlier detection) are effective in preventing clients from manipulating their contributions, even when they attempt to do so through multiple methods.
  • Topology Matters: The paper demonstrates that TRIP-Shapley can capture how network topology (e.g., star vs. line) influences contributions, a factor not present in centralized FL.

Key Results

  • Proximity to Shapley Value: TRIP-Shapley achieves small cosine distances to the ground-truth Shapley value, comparable to existing centralized FL methods, even under asymmetric network topologies. For instance, in a star topology, the cosine distance was as low as 0.007.
  • Scalability: In experiments with 100 clients, removing clients with high contributions (as identified by TRIP-Shapley) led to the largest drop in final model accuracy, while removing low-contribution clients resulted in the highest accuracy. This validates TRIP-Shapley's ability to accurately reflect client impact.
  • Correlation with Data Value: TRIP-Shapley scores correlate well with dataset quantity and quality. For example, a correlation of -0.864 was observed with noise amount on CIFAR-10.
  • Robustness to Dishonesty: When countermeasures were applied, dishonest clients' measured contributions were no higher than those of honest clients, effectively preventing manipulation. Without countermeasures, dishonest clients' contributions were significantly overestimated.

Limitations and Future Work

  • Computational Overhead on Clients: Calculating LCVs involves evaluating model performance for subsets of clients, which can be computationally intensive. The authors suggest using Monte Carlo approximations for Shapley values to mitigate this.
  • Communication Overhead: Exchanging both pre-training and post-training models can double communication volume. The paper suggests using gradient compression techniques.
  • Coordinator Dependency: The current method relies on a coordinator. Future work could explore scenarios without a coordinator, where clients might broadcast LCVs to each other, potentially using Byzantine fault-tolerant protocols.
  • Extension to Asynchronous DFL: The current work focuses on synchronous DFL. Extending TRIP-Shapley to asynchronous settings is a promising direction.
  • Fairer DFL Systems: The authors aim to leverage TRIP-Shapley for designing more equitable DFL systems, potentially through incentive mechanisms or adaptive weighting schemes.

Summary updated

Abstract

Federated learning (FL) enables multiple clients to collaboratively train machine learning models without sharing local data. In particular, decentralized FL (DFL), where clients exchange models without a central server, has gained attention for mitigating communication bottlenecks. Evaluating participant contributions is crucial in DFL to incentivize active participation and enhance transparency. However, existing contribution evaluation methods for FL assume centralized settings and cannot be applied directly to DFL due to two challenges: the inaccessibility of each client to non-neighboring clients' models, and the necessity to trace how contributions propagate in conjunction with peer-to-peer model exchanges over time. To address these challenges, we propose TRIP-Shapley, a novel contribution evaluation method for DFL. TRIP-Shapley formulates the clients' overall contributions by tracing the propagation of the round-wise local contributions. In this way, TRIP-Shapley accurately reflects the delayed and gradual influence propagation, as well as allowing a lightweight coordinator node to estimate the overall contributions without collecting models, but based solely on locally observable contributions reported by each client. Experiments demonstrate that TRIP-Shapley is sufficiently close to the ground-truth Shapley value, is scalable to large-scale scenarios, and remains robust in the presence of dishonest clients.
BibTeX
@misc{anada2025measuring,
  author = {Honoka Anada and Tatsuya Kaneko and Shinya Takamaeda-Yamazaki},
  title = {{Measuring Participant Contributions in Decentralized Federated Learning}},
  year = {2025},
  eprint = {2505.23246},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.23246},
}
Figure