Treating Client Dropout as Trial Attrition in Federated Low-Resource Language Training

Federated training lets communities that speak low-resource languages improve a shared language model without shipping their text across borders: each device trains a small piece of the model on local data and sends only that update to be averaged. The weak point is that devices drop out mid-round, and in practice they do not drop out at random. The phones on unstable connections or intermittent power are disproportionately the ones holding the least-represented languages. When the averaging server simply ignores whoever failed to report, the shared model drifts toward well-connected languages. Clinical trialists have a name for this failure: informative attrition, where non-random dropout destroys the balance that randomization was supposed to guarantee.

We propose to fix it with an idea borrowed from fair coin-flipping protocols in cryptography. In those protocols each party pre-commits a "defense value" so that aborting cannot change the result. Here, each client secret-shares a fallback update one round in advance. If it vanishes, the surviving clients add its fallback into the secure sum instead of omitting it, and the server never sees any individual contribution. We then test whether this helps with an actual randomized trial: volunteer devices are assigned at random to parallel federations that differ only in their dropout policy, so every arm faces the same real-world churn.

Core Claim

Under dropout that is correlated with language, replacing absent contributions with pre-shared fallbacks will reduce the accuracy gap between poorly connected and well-connected languages, relative to dropping or reweighting the absent clients. This holds without giving up secure aggregation, the property that the server learns only the sum of updates. The advantage should appear specifically when a language is held by very few clients. It should vanish when dropout is uniformly random, and that predicted null is part of the test.

Let $\Delta_i^t$ be client $i$'s prompt update in round $t$, let $m_i^t \in \{0,1\}$ indicate whether it reported, and let $n$ be the number of clients. The recovered aggregate is

$$\bar\Delta^t = \frac{1}{n}\sum_{i=1}^{n}\left[m_i^t\,\Delta_i^t + (1-m_i^t)\,\Delta_i^{t-1}\right].$$

Its deviation from full participation is $\frac{1}{n}\sum_i (1-m_i^t)(\Delta_i^{t-1}-\Delta_i^t)$, which is the change in one client's update over a single round. Drop-and-ignore instead loses the whole $\Delta_i^t$ of every absent client, and those losses accumulate systematically on the same languages. The construction mirrors the round-by-round fallback of An Optimally Fair Coin Toss. That protocol releases shares incrementally so that an aborting party can only choose between adjacent rounds' outcomes, which caps its influence at $O(1/r)$. Our analogue caps the damage from a dropout at one round of staleness.

How This Relates to Prior Work

The application setting comes from Breaking Physical and Linguistic Borders. That paper trains about 1.2M soft-prompt parameters on a frozen XLM-RoBERTa across clients, including MasakhaNEWS languages. It reports 6.9% higher accuracy than local monolingual fine-tuning and 479 MB of total communication. It assumes every client completes every round. It also names gradient inversion, meaning reconstruction of private text from updates, as an unaddressed risk and leaves secure aggregation to future work. We keep its model and data and add realistic dropout plus secure recovery.

Low-Resource Machine Translation through the Lens of Personalized Federated Learning constrains the design and also motivates it. Its MeritOpt-Drop variant pruned low-weight auxiliary languages and overfit badly on the low-resource target. That is direct evidence that losing participants is costly in this regime, even participants that look unimportant. MeritOpt itself runs on one machine, so it never encounters network dropout.

The recovery mechanism draws on the cryptography line. The many-party protocols in Fair Coin Flipping: Tighter Analysis and the Many-Party Case pre-distribute secret-shared defense values that survivors use to continue without the departed party. That idea is the template for our fallback shares.

Helium supplies a practical engineering reference point. It splits secure computation into small, retryable sub-protocols coordinated by an untrusted helper. It tolerates 40 failures per minute across 30 parties, but only on a synthetic 512×512 matrix-vector product and only against passive adversaries. Phoenix proves how many honest parties must be online in both of two consecutive rounds for state to carry across. Under computational security that number is at least one. It also requires that no honest party stays offline longer than $B$ rounds. We adopt the same structure: fallbacks age out after $B$ rounds, and a client absent longer than that is genuinely lost. This is a stated limitation, not something we solve.

Two papers shape the evaluation. High-Resource Methodological Bias in Low-Resource Investigations shows that down-sampled high-resource data misleads conclusions about low-resource behavior. We extend that warning from data to network conditions: synthetic churn may mislead in the same way that synthetic scarcity does, which is why the program includes a field trial.

The trial literature supplies the design. Multi-arm randomized studies such as the P.A.R.T.Y. Defence protocol and the AI-intervention trial in A large-scale randomized study of large language model feedback in peer review show how to isolate one intervention among many noisy factors. We borrow the multi-arm, cluster-randomized structure, not any clinical content.

Research Program

The work has two phases.

Phase 1: the mechanism and a controlled study. At the end of round $t-1$, each client splits its update $\Delta_i^{t-1}$ into threshold shares held by a small committee of peers. To keep cost down, it shares a seed-compressed version.

In round $t$, the present clients do two things. They submit masked current updates, and they also sum the shares they hold for absent clients and submit those sums. Because shares are linear, the server reconstructs only the single total. Individual fallbacks are never revealed, even when exactly one client is absent.

Strategic aborts get one targeted test. A malicious client that drops out to swing the average gains nothing beyond its own one-round-stale contribution, which is the coin-flipping guarantee restated for averaging. We test only this property and do not claim defense against corrupted shares, which Helium also leaves open.

Phase 1 runs on MasakhaNEWS and AfriSenti partitioned by language. We vary two things. The first is how many clients hold each language, from one to many. The second is how dropout depends on language, ranging from uniform to strongly tied to a connectivity class. That connectivity dependence is calibrated from Phase 2 pilot logs.

Phase 2: a field trial. We recruit volunteer devices across regions and stratify them by language and connectivity class. Within each stratum, devices are randomly assigned to one of several parallel federations, and each federation runs one dropout policy. Every arm therefore inherits the same real distribution of outages, latency bursts and battery deaths.

Interference rules out randomizing individual clients, since all clients in a federation share one model. We therefore randomize at the federation level and repeat the process in successive cohorts to obtain replicates.

The data on the devices is still the public benchmark partitions. What the trial makes realistic is the churn, and we state that plainly as the trial's scope.

The primary outcome is worst-language macro-F1 at a fixed communication budget. Secondary outcomes are the F1 gap between low- and high-connectivity languages, average F1, bytes per client, and wall-clock time per round.

Baselines

The naive reference is drop-and-ignore: federated prompt averaging as in Breaking Physical and Linguistic Borders, where the server averages whichever updates arrive. It runs on the identical backbone, prompt size, round count and dropout traces, with the same learning-rate sweep. Two reference points from the file frame the space but are not comparable numbers:

  • That paper reports a 6.9% accuracy gain over local fine-tuning on under-represented languages. The file gives no per-language absolute MasakhaNEWS score for it.
  • Centralized AfroXLMR-large reaches 92.6 average F1 on MasakhaNEWS (MasakhaNEWS). It uses a larger backbone with centralized data, so it serves as a ceiling.

On AfriSenti the centralized reference is NLNDE at 71.31 weighted F1 on Task A and 75.06 on Task B (AfriSenti-SemEval). Again it is a ceiling, not a direct competitor. We also run local monolingual prompt tuning as the floor.

The strongest simple competitor is inverse-probability-weighted aggregation, the standard trial fix for attrition. Each surviving client is upweighted by the inverse of its language's estimated reporting rate. Its weights get the same tuning budget as our method's staleness cap.

Timeout-and-retry, in the style of Helium's helper-coordinated retries, waits a fixed window and retries absent clients before giving up. We match its wall-clock budget to ours.

The ablation that isolates the key ingredient is a plaintext stale cache. The server simply stores each client's last update in the clear and substitutes it on dropout. This removes all cryptography while keeping substitution. If the cache matches our accuracy, the accuracy gain comes from substitution and the secret sharing buys only privacy. If the cache also matches our privacy-free cost, the cryptographic layer is justified only where privacy matters.

Beating the field means three things. First, a worst-language F1 improvement over both drop-and-ignore and IPW that holds across trial replicates, in the regime of few clients per language with connectivity-correlated dropout. Second, no loss in average F1. Third, a communication overhead that we report against the 479 MB reference.

Why not just…?

Why not just reweight surviving clients, as trialists do for attrition? This is the real threat. When a language is held by many clients, IPW can restore balance with no cryptography at all, and we expect it to tie with our method there. It fails when every client holding a language is absent in the same round, because then there is no same-language update left to upweight. That is common precisely when a language has one or two contributors. The client-count sweep is designed to locate the crossover, and if IPW wins everywhere we will report that.

Why not just cache last updates on the server? It works mechanically, and it is our ablation. It requires the server to see individual prompt updates, which reopens the gradient-inversion exposure that Breaking Physical and Linguistic Borders flags. Under secure aggregation the server has nothing to cache, and pre-shared fallbacks are how substitution survives encryption.

Why not just wait and retry? Retrying buys completeness at the cost of latency. A retry also does not help a client that is offline for the whole window, which is the typical failure on intermittent power. The matched wall-clock comparison measures this tradeoff directly. If retry reaches equal worst-language F1 within a tolerable round time, the recovery protocol is unnecessary for crash-only churn.

Why not just simulate churn instead of running a trial? Simulated dropout is the network analogue of down-sampled English. The field arms test whether the ranking of policies under calibrated simulation matches the ranking under real outages.

Decisive Checks

The idea is supported only if three results hold together:

  • Recovery beats both drop-and-ignore and IPW on worst-language F1 when dropout is tied to connectivity and languages have few holders.
  • It shows no advantage under uniform random dropout, which serves as a negative control.
  • The policy ranking from the field trial agrees with the simulation.

If the plaintext cache matches recovery, we narrow the claim to privacy-preserving substitution. If IPW matches recovery across the client-count sweep, we drop the claim entirely.