In Federated Learning (FL), the standard approach is FedAvg: clients train locally, and the server averages their updates. But when clients have very different data (non-IID), their updates can "conflict," pointing in opposing directions. A new wave of "conflict-averse" aggregators (like FedFV or FedMGDA+) aims to fix this by finding a direction that satisfies everyone.
This paper takes a skeptical look at the "conflict rationale." By isolating the effects of one such aggregator, UPGrad, the authors find that the story is much more complicated than "conflict is bad, and fixing it is good."
TL;DR
Conflict-averse aggregation in Federated Learning is often justified by the need to resolve gradient conflict, but this paper shows that "conflict" is actually four distinct quantities that don't always move together. Surprisingly, the accuracy gains from these complex methods can often be matched by simply using uniform client weights, suggesting the benefit comes from implicit reweighting rather than resolving conflict.
What Problem Does This Paper Address?
The core challenge in FL is client drift. Because clients have heterogeneous data, their local models drift toward local optima that are globally inconsistent. Researchers have theorized that averaging these "conflicting" updates cancels out useful signals.
To solve this, several recent papers have adapted multi-objective optimization techniques to FL. These methods try to ensure the global update is "non-conflicting"—meaning it doesn't increase the loss for any individual client (at least to a first-order approximation). However, these methods are usually evaluated only on final accuracy. This paper asks: Does the "conflict rationale" actually hold up? Does resolving conflict at the server actually lead to the benefits we see, and are we measuring the right things?
Methods
The authors focus on UPGrad (Universal Projected Gradient), an aggregator that projects client updates onto a "dual cone" to ensure no client is harmed. To truly understand what UPGrad does, they use a rigorous diagnostic protocol:
- Direction-Only Comparison: They rescale UPGrad’s output to match the norm of the standard weighted mean. This prevents UPGrad from acting as a "hidden learning rate multiplier," isolating the effect of the update direction.
- The Four Quantities: They break down the "conflict rationale" into four measurable metrics:
- Conflict: How much do client updates actually oppose each other?
- Feasibility: Does the standard weighted mean actually violate the non-conflicting condition?
- Faithfulness: Do the updates the server receives actually represent the clients' true gradients after a full local epoch?
- Displacement: How much does the aggregator actually rotate the update?
- Ablation Controls: They compare UPGrad against "uniform weighting" (which ignores conflict but changes weights) and "matched random rotation" (which rotates the update by the same amount as UPGrad but in a random direction).
Datasets
The study uses three primary regimes:
- Dirichlet CIFAR-10: 20 clients with label skew controlled by a Dirichlet distribution ($\alpha=0.5$).
- Disjoint-class CIFAR-10: 5 clients with entirely different classes—a "maximum conflict" scenario.
- Dirichlet Oxford-IIIT Pet: 20 clients fine-tuning a ResNet-18.
Key Insights and Ideas
- The Quantities "Come Apart": High conflict doesn't necessarily mean the mean update is "infeasible" (harmful). In the "Disjoint" split, conflict was at its peak, yet the weighted mean was almost always feasible, and UPGrad provided no accuracy gain.
- The Faithfulness Gap: Because clients perform multiple local SGD steps (an epoch), the "pseudo-gradient" they send back to the server is a poor proxy for their true gradient at the current global model. The "non-conflicting" guarantee holds for the transmitted updates but often fails for the clients' actual losses.
- Reweighting is the Secret Sauce: In both CIFAR-10 and Pets, simply weighting all clients equally (uniform weights) matched or even outperformed UPGrad. UPGrad implicitly gives more weight to smaller clients; the authors argue this reweighting, rather than "conflict resolution," is likely driving the observed gains.
- Fragile Gains: The gain on the Pets dataset (+0.52 pp) completely vanished when the authors changed the data protocol to use all available samples, suggesting these small gains are highly sensitive to experimental setup.
Key Results
The quantitative results show that while UPGrad can provide a statistically significant boost, the effect size is small and highly conditional.
As seen in the figure above and the table below, the gains are most prominent in the Dirichlet CIFAR-10 setting (+0.78 pp). However, in the "Disjoint" setting—specifically designed to maximize conflict—the performance actually trended slightly negative (-0.23 pp).
Limitations and Future Work
The authors acknowledge several limitations:
- Scale: The study is relatively small (up to 20 clients and 150 rounds).
- Scope: They only tested one specific non-conflicting operator (UPGrad), though it is a representative one.
- Local Training: The paper focuses on server-side aggregation. The authors suggest that since the "faithfulness" of updates is so low, future effort might be better spent improving local training rather than complex server-side math.
The paper concludes with a call for better reporting in FL research: evaluations of new aggregators should report conflict, feasibility, and reweighting controls to ensure we aren't crediting "conflict resolution" for what is actually just a change in client weighting.
Summary updated