Word sense disambiguation means deciding which dictionary meaning a word carries in context: bank as a lender or bank as a riverside. Systems that do this well learn from large hand-labeled corpora, and no such corpora exist for regional dialects. The text that would help most, everyday writing in Bavarian, Alemannic or Singapore English, mostly sits on people's phones and in private chats, where it can be neither labeled nor collected. This proposal rests on one observation. For this task the labels are public and only the inputs are private. The label space is a dictionary of sense definitions (glosses) that a server can hold openly. A model that compares a sentence to those definitions can therefore label each user's text on the user's device, with no human involved.
We propose federated self-training around a gloss-informed disambiguation model. Federated learning means training a shared model by sending it to the data and averaging the updates, without moving the raw text. A server holds the encoder for dictionary definitions and keeps it fixed. Each client labels its own dialect sentences with whatever senses the current model is confident about, then trains a small adapter on the lower layers of the sentence encoder, and the server averages those adapters. The bet is that the frozen definitions act as a common target that holds dialect-diverse clients together. A second bet is that pooling across many users is the only way to collect enough examples of rare senses, because any single user sees too few of them to learn from.
Core Claim
The claim has two parts, and each can be tested on its own. First, a fixed public label space removes the problem that clients hold no labels, which standard federated learning assumes away. FedHealth (FedHealth), for example, federates shared layers and personalizes classifier heads on clients that already hold clean activity labels. Here clients hold only raw dialect text, and the "head" is the server's table of gloss embeddings. Second, the benefit of federating should land mostly on less frequent senses. When each client self-trains alone, confidence thresholds favor the senses it already predicts well. This is the sense-prior bias that Agirre et al. showed dominates automatically harvested WSD data. Federation adds evidence from many clients, and federated counting lets the server see which senses the pseudo-labels are starving.
Both parts rule something out. If federated training helps all senses about equally, the gain is plain extra data and the rare-sense story is wrong. If simply mapping dialect spellings to standard forms closes the gap, the federated machinery is unnecessary. The experiments below test both.
Research Program
The base model is the gloss-informed bi-encoder of Blevins and Zettlemoyer. It encodes contexts and definitions with two separate BERT encoders and scores a sense by the dot product of the two embeddings. On standard English it reaches 79.0 F1 overall, 94.1 on most frequent senses and 68.9 on unseen senses. It supplies the method. We keep its structure and change where each half lives. We chose it over the single shared ranking classifier of Choubey and Huang because separating the definition encoder is exactly what lets the server keep it fixed. Choubey and Huang's evidence that sharing parameters across words roughly doubles F1 on words with one or two training instances (25.8 vs. 13.8) is a reason to expect pooling rare evidence to pay off.
On each client the context encoder carries low-rank adapters (LoRA, small trainable weight corrections added to a frozen network). We place them on transformer layers 0–3, following DADA's probing finding that dialect morphosyntax is handled mostly in the lowest layers. The server sends the global adapter and a precomputed table of gloss embeddings. Clients receive the whole table rather than asking for embeddings of particular words, because a per-word request would reveal their vocabulary. Bandwidth for the table, reduced in precision and dimension, is reported as a cost.
Each client scores the ambiguous words in its local text and keeps a pseudo-label only when the margin between the top two senses clears a threshold. This mirrors Train-O-Matic's confidence-gap filter (Pasini and Navigli) and the 0.6 confidence cutoff of bilingual bootstrapping (Khapra et al.). The new piece is that the threshold is set per sense from global statistics. Each round, clients report how many pseudo-labels they produced per sense, combined through secure aggregation, which is federated analytics in the sense of this survey. The server then broadcasts per-sense thresholds:
$$\delta_c = \delta_0 \left(\frac{\hat n_c}{\bar n}\right)^{\gamma}.$$
Here $\hat n_c$ is the aggregated pseudo-label count for sense $c$, $\bar n$ is the mean count over senses of the same word, $\delta_0$ is a base margin, and $\gamma \ge 0$ controls how strongly over-produced senses are throttled. A rare sense gets a lower bar and a dominant one a higher bar. Setting $\gamma=0$ recovers the fixed threshold. Clients train their adapters on the kept pseudo-labels with the usual cross-entropy over candidate glosses, and the server combines them with Federated Averaging weighted by pseudo-label count.
The program has three parts. The first is building dialect WSD test sets, which do not exist today. The second is the federated system itself. The third is a set of controlled analyses that vary the number of clients, how dialects are mixed across clients, and whether the gloss encoder is frozen. We do not claim to handle senses that exist only in the dialect and have no gloss. Such cases are scored as abstentions and counted separately, so that this limit is measured rather than hidden.
Baselines
The naive reference point is the standard-language bi-encoder applied to dialect text with no adaptation. On standard English it scores the 79.0 F1 reported by Blevins and Zettlemoyer. We found no reported result on Lacuna for this or any WSD model on dialectal text, so we measure its dialect drop ourselves. We expect a large drop, by analogy with Multi-VALUE's 25.4% BERT degradation on Singapore English question answering. Every other system starts from this checkpoint.
The strongest existing method that uses no private data is centralized training on public and synthetic dialect text. On English this means fine-tuning the bi-encoder on SemCor rewritten into dialects with Multi-VALUE, and adding DADA-style rule adapters. On German it means centralized self-training on public Bavarian and Alemannic Wikipedia, with the same pseudo-labeling rule. It is the strongest competitor because it needs no federation and no trust in clients. Its English data-augmentation analogue is reported to give 2.7 F1 cross-dialect at a 1.2-point cost on standard English (Multi-VALUE).
A second strong competitor is normalize-then-disambiguate. It maps dialect spellings to standard words using lexicons induced with LaBSE bitext mining and awesome-align (Artemova and Plank), then runs the unmodified bi-encoder. The third competitor is per-client test-time adaptation with no federation: SHOT (Nguyen et al.), plus local-only self-training with our pseudo-labeling rule.
The upper bound is centralized self-training on the pooled client data, which is what federation would match if moving the data were allowed. Two ablations isolate the proposal's ingredients. Setting $\gamma=0$ isolates the count-calibrated thresholds. Training the gloss encoder federatedly instead of freezing it isolates the shared anchor.
All systems get the same unlabeled data where applicable, the same number of local gradient steps in total, and the same budget of 20 hyperparameter configurations. The thresholds $\delta_0$ and $\gamma$ are tuned on a small labeled development split. The rule-adapter count for DADA is tuned on the same split. Beating the strongest baseline means a statistically significant gain in less-frequent-sense F1 averaged over dialects, measured by paired bootstrap over test instances. The gain must come with no loss in overall F1 and at most a 0.5-point drop on standard-language F1, a smaller interference cost than Multi-VALUE's 1.2.
Why not just…?
Why not just normalize dialect spellings to the standard language and run the existing model? This is the most serious threat, and we treat it as one. Artemova and Plank report 85–95% alignment accuracy for low-to-mid frequency words, and much dialect variation is spelling (Jahr → Joahr, Joar, Joa, Joo). Their own finding limits it, though. Alignment degrades on high-frequency function words, and mappings are one-to-many, so normalization must itself guess. It also does nothing about grammatical differences, which Multi-VALUE shows are a major source of failure. We run normalization as a baseline and also as preprocessing under every federated arm. If normalization alone gets within noise of federated self-training on the German test set, we will report that the spelling problem is the whole problem for Bavarian and Alemannic. The federated contribution would then be restricted to the morphosyntactic English setting.
Why not just adapt each device locally at test time? SHOT needs large, coherent batches to estimate its clustering statistics, and its reported average gain is a modest 0.98 on GLUE-style tasks. EMEA (Wang et al.) slows inference from about 250 to 9 examples per second. More fundamentally, a single user's text holds too few examples of any rare sense. DIALECTBENCH (Faisal et al.) documents that fine-tuning on small dialect samples can even hurt relative to zero-shot transfer. Our scaling experiment over client count tests this directly. If local-only training matches federation once clients exceed a modest size, federation is not needed.
Why not just train centrally on public and synthetic dialect data? For English morphosyntax this may be enough, since that is what DADA and HyperLoRA (Xiao et al.) achieve. Synthetic rules, however, do not model spelling or vocabulary, and public dialect Wikipedia is small: 17,000 mined parallel sentences for Bavarian. The comparison against the pooled-client upper bound settles how much in-distribution text is worth. If centralized public training gets within one F1 point of that bound, the case for touching private data at all collapses, and we would say so.
Why not just prompt a large language model zero-shot? Sainz et al. show language models carry real sense knowledge. They reach 86.8 F1 on adjectives with domain-level inventories but only 57.0 on verbs, and running such models on a phone is impractical. We include an NLI-based zero-shot run as a reference on the coarse-domain version of our tests. We do not treat it as a deployable competitor.
Proposed Experiments
The controlled English experiment rewrites SemCor and the standard all-words suites (Senseval-2/3, SemEval-2007/2013/2015) into dialects using Multi-VALUE's meaning-preserving rules. Any target word that a rule itself rewrites is excluded, so gold senses stay valid. SemCor's labels are then discarded. Its rewritten text is split across 200 simulated users with Zipfian sizes. We vary the mix from one dialect per client to clients that blend several dialects, following DADA's view that dialects overlap. Results are F1 on all, most-frequent and less-frequent senses for each dialect, F1 on original standard English, bytes communicated, and on-device training time.
The natural-text experiment uses German, Bavarian and Alemannic. Using the mined parallel sentences from Artemova and Plank, we sense-tag the German side with a multilingual pipeline in the style of Train-O-Matic and the semi-automated construction pipeline. We then carry the labels over to the aligned dialect words, the second-language bridge of Wang. Native speakers verify about 2,000 instances per dialect as the test set, which we score at both fine-grained and coarse domain-inventory granularity. Training clients are Bavarian and Alemannic Wikipedia partitioned by article. Pages used for testing are held out.
The decisive checks are four. The first plots less-frequent-sense F1 against the number of participating clients for federated versus local-only training. The claim predicts the gap widens with client count and concentrates on rare senses. The second compares $\gamma=0$ with tuned $\gamma$; the calibration must raise less-frequent-sense F1 without lowering most-frequent-sense F1 below the unadapted model. The third compares frozen against federatedly trained gloss encoders under the most mixed client partition. The anchoring claim predicts the frozen version degrades less as mixing increases. The fourth reports, on the natural German data, the head-to-head against normalize-then-disambiguate and against the pooled-centralized bound. That comparison determines whether federation adds anything beyond spelling repair, and how much of the private-data value it recovers.
References
- FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare
- Unsupervised WSD based on Automatically Retrieved Examples: The Importance of Bias
- Moving Down the Long Tail of Word Sense Disambiguation with Gloss Informed Bi-encoders
- One Classifier for All Ambiguous Words: Overcoming Data Sparsity by Utilizing Sense Correlations Across Words
- DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules
- Train-O-Matic: Large-Scale Supervised Word Sense Disambiguation in Multiple Languages without Manual Training Data
- Together We Can: Bilingual Bootstrapping for WSD
- A Survey on Federated Analytics: Taxonomy, Enabling Techniques, Applications and Open Issues
- Multi-VALUE: A Framework for Cross-Dialectal English NLP
- Low-resource Bilingual Dialect Lexicon Induction with Large Language Models
- Harnessing Test-time Adaptation for NLU tasks Involving Dialects of English
- Efficient Test Time Adapter Ensembling for Low-resource Language Varieties
- DIALECTBENCH: An NLP Benchmark for Dialects, Varieties, and Closely-Related Languages
- Task-Agnostic Low-Rank Adapters for Unseen English Dialects
- What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories
- Semi-Automated Construction of Sense-Annotated Datasets for Practically Any Language
- Automatic Acquisition of English Topic Signatures Based on a Second Language