Most of the world's languages have almost no labeled data, so their tools are usually built by training on a related language that does have labels and hoping the model carries over. A standard way to encourage that carry-over is adversarial domain adaptation. A second network, the discriminator, tries to guess which language a sentence's internal representation came from, and the main model is trained to fool it. The result is that every language looks the same inside the model. This is a blunt instrument for languages, because languages are not equidistant. isiXhosa and isiZulu are close cousins, and both are far from Hausa. Forcing all three into one indistinguishable cloud throws away the family structure that makes transfer work in the first place.
We propose tiered adversarial alignment. Each branch point of a language family tree gets its own discriminator, and each discriminator only tries to tell apart the languages directly below it. Alignment is strong among close relatives and weak, or absent, between distant families. The design borrows the logic of a journal's editorial board. Fine-grained judgments are made locally by people who know the subfield, and only broad standards are enforced at the top. The claim is concrete. On zero-shot named-entity recognition across 20 African languages, tiered alignment should beat both flat adversarial alignment and the current best practice of choosing the most related source language. The gains should be largest exactly where the nearest labeled relative is far away. If a randomly shuffled tree of the same shape works just as well, the claim is false.
What exists and where the gap is
Three lines of prior work motivate this. MasakhaNER 2.0 built human-annotated NER data (tagging person, place, organization and date names) for 20 Sub-Saharan languages. It showed that transferring from a linguistically and geographically close African language beats English by about 14 F1 points on average. It also found that co-training on the top two predicted source languages adds roughly 3 F1. That work uses plain supervised fine-tuning with no alignment of representations.
KINNEWS and KIRNEWS showed that mutual intelligibility alone carries a Kinyarwanda-trained classifier to 67.54% on Kirundi. That is well below the within-language numbers, which leaves a gap that alignment could close. Together, these papers establish that relatedness matters, but neither uses it inside the training objective.
The method line constrains and supplies our design. Taxonomy-Structured Domain Adaptation (TSDA) proves that perfect uniform adversarial alignment and preservation of an informative domain taxonomy are mathematically incompatible. Its fix adds a "taxonomist" network that reconstructs a given distance matrix from the representations. With that fix it reaches 82.4% versus 66.9% for flat adversarial training (DANN) on a bird benchmark whose domains are organized by the biological (NCBI) taxonomy. This is the closest existing work.
We differ in two ways. First, TSDA keeps a single discriminator that aligns everything and a separate player that pulls structure back in. We instead restrict what each discriminator is allowed to compare, so alignment strength is set per level of the tree rather than negotiated globally. Second, TSDA has never been applied to languages. Its authors also flag the open risk that a pre-specified tree may not match the task. We test that risk directly by comparing a genealogical tree, a geographic tree, and a tree built from measured pairwise transfer.
Hierarchical adversarial training for dialogue labeling also stacks discriminators, but it does so across levels of the input (utterance versus whole dialogue), not across domains. We reuse its gradient-reversal machinery. A gradient reversal layer is a layer that passes features forward unchanged but flips the sign of the gradient on the way back, so the encoder learns to fool the discriminator.
The editorial-governance material in the file supports only a structural analogy. The IEEE publication-information pages (Transactions on Information Theory, Systems, Man, and Cybernetics) document tiered review, in which local experts evaluate submissions within their area and top editors enforce scope. Guo et al. found that LLM agent teams with a designated leader finish tasks faster, and that organizational structures refined through their iterative Criticize-Reflect procedure beat hand-designed ones. We take one lesson from this. Structure should be revisable from evidence rather than fixed a priori, and that motivates the learned-tree condition below. We do not claim anything about human editorial boards.
Core Claim
Let $h = E(x)$ be the encoder representation of a sentence $x$. For each internal node $v$ of a language tree, let $D_v$ be a discriminator that sees only sentences from languages under $v$. Its job is to predict which child subtree each sentence came from. The training objective is
$$ \min_{E,F}\max_{\{D_v\}} \; \mathcal{L}_{\text{NER}}(F(E(x)), y) \;-\; \sum_{v} \lambda_{d(v)}\, \mathcal{L}_{D_v}\big(D_v(E(x)), c_v(x)\big), $$
In this objective, $F$ is the NER tagging head. $y$ is the gold label sequence, available for source languages only. $c_v(x)$ is the child of $v$ that contains $x$'s language. $\lambda_{d(v)}$ is an alignment weight that depends on the depth $d(v)$ of node $v$, and it increases toward the leaves.
The prediction follows from TSDA's incompatibility result. Flat alignment should hurt most for targets that sit in dense clusters of relatives, such as the Nguni and Bantu languages, because it erases useful cluster identity. Tiered alignment should help most for targets whose closest labeled relative is several branch points away. For those targets, best-source selection has no good source to pick.
Research Program
We run leave-one-language-out over MasakhaNER 2.0. Each held-out target contributes only unlabeled text, and all remaining languages supply labeled data. Target news text comes from the same sources as the benchmark.
The backbone is AfroXLM-R, which MasakhaNER 2.0 reports as its strongest model. Before any alignment, the backbone first receives language-adaptive fine-tuning: continued masked-language-model training on the target's unlabeled text. MasakhaNER reports over 60% improvement from this step on Amharic, so every method starts from this stronger point.
We build three trees over the 20 languages. The first is a genealogical tree from standard language classification. The second is a geographic clustering tree. The third is an empirical tree built by agglomerative clustering of the pairwise zero-shot transfer matrix. That matrix is computed only among the source languages in each fold, so the target never leaks into it. The empirical tree is the "revisable board": structure inferred from observed transfer rather than taken from a reference book.
As a second, much smaller test, we run the Kinyarwanda→Kirundi pair from KINNEWS/KIRNEWS. There we embed the pair inside a tree with more distant Bantu and non-Bantu relatives, trained on topic classification, to check that the effect is not specific to NER.
Baselines
The naive reference point is the same language-adapted backbone fine-tuned on the pooled labeled data of all source languages, with no discriminator. This is the pure "more diverse data" answer. We run it with the same steps and the same hyperparameter search budget as every other method: an identical number of trials over learning rate and warmup.
The strongest existing method is MasakhaNER 2.0's recipe. It picks sources with LangRank, a ranker that combines geographic distance, genetic distance and entity overlap, then co-trains on the top two predicted languages. The paper reports roughly 14 F1 over English transfer on average and about 3 additional F1 from co-training. It also reports 85.5 F1 for isiXhosa→isiZulu, and says zero-shot transfer from the best source matches annotating 500 target sentences. We will rerun this recipe inside our own folds, on the same backbone and budget, rather than copy numbers. Beating it means a higher F1 averaged over the 20 folds, with a paired bootstrap confidence interval that excludes zero. It also means a clear win on at least the targets whose best source is outside their immediate branch.
Flat DANN, with one discriminator over all languages, isolates whether any alignment helps. TSDA run with our genealogical tree tests whether our per-node design beats the existing structured method. No NER or text result for TSDA or DANN was found on Lacuna; the reported comparison is 82.4 versus 66.9 on the bird benchmark.
The key ablation replaces each tree with a random tree of identical shape. Every discriminator, parameter count and alignment weight stays the same; only the grouping changes. A second ablation sets all $\lambda_{d(v)}$ equal, to test whether depth-graded strength matters.
Why not just…?
Why not just pick the closest source language and co-train on the top two? This is the real threat. MasakhaNER 2.0 shows it recovers most of the benefit of relatedness at zero extra complexity. Where a target has a very close labeled sibling, such as isiZulu with isiXhosa, we expect little or no gain over it, and we will say so. Our method earns its place only if it improves targets whose relatives are distant. It could do so because it uses all sources in a graded way instead of discarding all but two. We report per-target gains against tree distance to the nearest source. If the gain curve is flat, the recipe wins and the proposal fails.
Why not just pool every language and rely on diversity? Diversity does help generalization, as the survey by Luo et al. and BLOOM's data ablations in Le Scao et al. argue. But MasakhaNER 2.0's own result that a well-chosen single source beats English shows that source choice, not just variety, drives transfer. The pooled baseline measures directly how much of our gain is merely exposure to more languages.
Why not just use flat adversarial alignment, or TSDA as published? TSDA's incompatibility theorem predicts that flat alignment damages informative structure, and its bird-benchmark gap is evidence of that. TSDA itself is a legitimate competitor, and if it matches our tiered discriminators, the honest conclusion is that the linguistic tree is what matters, not our architecture. The head-to-head comparison under the same tree settles that question.
Why not just do language-adaptive fine-tuning and stop? It is cheap and strong. That is why it is included in every condition, including the baselines, so any gain we report is on top of it.
Proposed Experiments and Decisive Checks
The primary metric is entity-level F1 on each held-out MasakhaNER 2.0 language, averaged over three seeds. We also express each gain in "equivalent annotated target sentences", obtained by fine-tuning on 100, 250 and 500 target sentences and interpolating. This anchors results to MasakhaNER 2.0's 500-sentence parity point.
On KIRNEWS the metric is accuracy, against the reported 67.54% BiGRU zero-shot figure and our own rerun of the transformer baselines.
Three outcomes are decisive. First, the shuffled-tree ablation must lose clearly to the real trees; otherwise structure is not the active ingredient. Second, flat DANN should fall below the pooled baseline on dense-cluster targets, as the incompatibility theorem predicts, while tiered alignment should not. Third, the empirical transfer tree should at least match the genealogical tree. If the reference-book tree wins, fixed linguistic classification is a sufficient source of structure, and the "revisable board" idea adds nothing. If it loses, TSDA's worry about mismatched taxonomies is real for languages.
A result where tiered alignment beats the source-selection recipe only on distant-relative targets would still be useful. It would tell practitioners working on isolated languages, which is most of the long tail, which tool to reach for.
References
- MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition
- KINNEWS and KIRNEWS: Benchmarking Cross-Lingual Text Classification for Kinyarwanda and Kirundi
- Taxonomy-Structured Domain Adaptation
- Unsupervised Domain Adaptation for Dialogue Sequence Labeling Based on Hierarchical Adversarial Training
- IEEE Transactions on Information Theory publication information
- IEEE Transactions on Systems, Man, and Cybernetics publication information
- Embodied LLM Agents Learn to Cooperate in Organized Teams
- MasakhaNER: Named Entity Recognition for African Languages
- A Survey on Efficient Large Language Model Training: From Data-centric Perspectives
- What Language Model to Train if You Have One Million GPU Hours?