A morphological tagger labels every word with its grammatical features: part of speech, gender, number, case, tense, mood, attached clitics, and so on. Building one for a low-resource variety such as Egyptian Arabic usually means borrowing from a related high-resource variety such as Modern Standard Arabic. The standard way to borrow is adversarial domain adaptation. A small classifier, the "discriminator," tries to tell from the shared encoder's representation which variety a sentence came from. The encoder is trained to fool it, so the representation becomes variety-agnostic. This works, but it is blunt. It removes everything that distinguishes the two varieties. That includes the very signals needed to tag constructions that exist in only one of them, such as Egyptian circumfix negation mA…š. It also says nothing about whether, say, an Egyptian accusative ends up near a Standard Arabic accusative or near a Standard Arabic genitive.
We propose to replace the single marginal discriminator with one discriminator per grammatical feature. Each is conditioned on the tagger's predicted distribution for that feature, restricted to the feature values both varieties share. Values that exist in only one variety are left out of the alignment entirely. In plain terms, the model is asked to make "a predicted feminine plural in Egyptian" look like "a predicted feminine plural in Standard Arabic." It is not asked to make Egyptian look like Standard Arabic overall, and it is never asked to erase Egyptian-only morphology. The claim is that this factorized, value-restricted conditional alignment beats marginal adversarial training most on two things. The first is syntactically governed features (case, number, mood), where marginal alignment lets value clusters cross. The second is dialect-only features, which marginal alignment actively damages.
Where this sits
The closest work is Adversarial Multitask Learning for Joint Multi-Feature and Multi-Dialect Morphological Modeling. It trains a shared Bi-LSTM with one output head per morphological feature, plus a single dialect discriminator behind a gradient reversal layer. That layer passes activations forward unchanged and flips the sign of gradients flowing back, so the encoder learns to fool the discriminator. The paper reports up to 10% relative error reduction, and 71.1% accuracy with 2,000 Egyptian tokens against 29.7% for an Egyptian-only model. Its discriminator sees only the encoder state, so it aligns the marginal distribution of representations. It never conditions on any feature prediction. Our proposal keeps its architecture, data, and shared output heads, and changes only what the adversary is allowed to look at and what it is allowed to align.
The conditioning mechanism comes from Conditional Adversarial Domain Adaptation. That paper feeds the discriminator the outer product of features and classifier probabilities, and weights examples by prediction confidence. On Office-31 this raised average accuracy from 82.2% (marginal adversarial training) to 87.7%. That paper has a single label space for image classification. The obvious transplant, a single discriminator conditioned on the concatenated or joint bundle of all feature heads, runs into the sparsity problem that already plagues "mega-tag" taggers. It also aligns the joint distribution of representation and label. That is exactly wrong when one domain has label values the other lacks.
Two kinds of prior work constrain the design. The invariance–discriminability literature (Chen et al., Wang, Zhang et al.) reports that forcing full invariance collapses fine-grained class boundaries and strips domain-specific cues that predictions depend on. This is why we refuse to align dialect-only values. Adversarial Adaptation of Synthetic or Stale Data shows that gradient reversal in NLP encoders can overshoot into "anti-domain" representations. It fixes this by training the encoder toward a uniform 50/50 discriminator output, called random prediction training. We adopt that objective inside each per-feature discriminator, and we test it separately so that any gain cannot be attributed to it alone.
Core mechanism
Notation:
- $h_t$ is the shared encoder state for word $t$.
- $p_{k,t}$ is the tagger's predicted distribution over values of feature $k$.
- $S_k$ is the set of values of feature $k$ attested in both varieties. This set is computed from the small labeled Egyptian set together with the analyzer inventories of SAMA/CALIMA described in A Morphological Analyzer for Egyptian Arabic.
- $\tilde p_{k,t}$ is $p_{k,t}$ restricted to $S_k$ and renormalized.
Feature $k$ gets its own discriminator $D_k$, which receives $h_t \otimes \tilde p_{k,t}$. When this outer product is too large, it is compressed by random projection, as in the conditional adversarial paper. Tokens are weighted by $w_{k,t}$. This weight combines prediction confidence, measured by the entropy of $\tilde p_{k,t}$, with the probability mass the tagger places on shared values. A token the tagger thinks is Egyptian-only negation therefore contributes almost nothing to the negation-feature adversary.
The training objective is
$$\mathcal{L} = \sum_k \mathcal{L}^{\text{tag}}_k \;+\; \lambda \sum_k \sum_t w_{k,t}\, \mathrm{KL}\!\left(u \,\|\, D_k(h_t \otimes \tilde p_{k,t})\right),$$
where $\mathcal{L}^{\text{tag}}_k$ is the supervised cross-entropy for feature $k$ on labeled tokens and $u$ is the uniform two-way distribution, following random prediction training. Each $D_k$ is separately trained to classify domain. Source tokens are reweighted by estimated target value proportions, so that differing frequencies of shared values do not masquerade as domain signal.
Research program
The work has three parts. First, we reimplement the Zalmout and Habash tagger on the Penn Arabic Treebank (Standard Arabic) and the LDC Egyptian Arabic Treebank. The aim is to reproduce their low-resource curve at 2k, 4k, 8k, and 16k labeled Egyptian tokens, with unlabeled Egyptian text supplied to the adversary. Second, we add the per-feature conditional discriminators. We construct the shared-value sets with an explicit audit of which values are excluded, including the Egyptian negation and future-marker clitics and other values the analyzer lists as dialect-only. Third, we run a diagnostic study. For each feature, we probe how separable the two varieties' representations are within each predicted value, and how often a source value's cluster lies nearest a different target value. This tests directly whether marginal alignment causes value crossing and whether conditioning removes it.
To show the effect is not Arabic-specific, we repeat the method on Universal Dependencies cross-lingual pairs, as in Neural Factor Graph Models for Cross-lingual Morphological Tagging: Spanish→Portuguese, Danish→Swedish, Russian→Bulgarian, and Finnish→Hungarian, with 100 and 1,000 target sentences. Here the shared-value sets come from UD feature inventories. Schema mismatches noted in Marrying Universal Dependencies and Universal Morphology mean feature-value sets must be harmonized before $S_k$ is computed. We will report how much the result depends on that harmonization.
Baselines
The naive reference points are an Egyptian-only tagger and a source-only tagger, the latter trained on Standard Arabic and applied to Egyptian. Both use the same encoder. The paper reports 29.7% for Egyptian-only at 2k tokens, and our reimplementation should land near that figure before anything else is trusted.
Pooled multitask training without any adversary isolates what adversarial alignment adds at all. Zalmout and Habash report that their adversary improves on this setup by up to 10% relative error. If our reproduction shows no such gap, the premise that alignment matters here is weak and we will say so.
The strongest existing method is Zalmout and Habash's adversarial multitask tagger, with its reported 71.1% accuracy at 2,000 Egyptian tokens (paper). We run it on identical splits, embeddings, analyzer features, and unlabeled target text. It gets the same budget for tuning the adversarial weight and learning rate as our method: the same number of trials, chosen on the same Egyptian development set. Beating it means higher full-bundle accuracy at 2k and 4k tokens across five seeds, by a margin larger than seed variance. It also means no loss on Standard Arabic and a clear gain on dialect-only tokens.
On the UD pairs, the reference is the neural factor graph. For Portuguese with 100 sentences, it reports macro-F1 of 74.07 against 10.33 for its baseline. We report this number but compare primarily against our own reimplemented marginal-adversary and non-adversarial versions on that encoder. For the Danish/Swedish, Russian/Bulgarian, and Finnish/Hungarian pairs, the file contains no per-pair figure: no reported result found on Lacuna for those pairs. Our own reimplementations will set those reference points.
Three ablations isolate the key ingredient. The first is marginal adversary plus random prediction training, which separates the conditioning from the training-objective fix. The second is a single discriminator conditioned on the full concatenated bundle, which separates per-feature factorization from conditioning per se. The third is per-feature conditioning without the shared-value restriction, which isolates the claim about not erasing dialect-only morphology. If the third matches the full method, the shared-value idea is unnecessary and we will drop it from the claim.
Why not just…?
Why not just fix gradient reversal with random prediction training and keep the marginal adversary? Kim et al. showed this objective change alone matters in spoken language understanding. It is a real threat, because some of the instability in Zalmout and Habash's adversary may be exactly this overshoot. That is why marginal-plus-uniform is a first-class ablation, tuned with equal effort. Our prediction is that it recovers some aggregate accuracy but not dialect-only tokens. A marginal adversary still pushes negation-bearing Egyptian tokens toward Standard Arabic, which has no such clitic. If it also recovers dialect-only tokens, conditioning is not the operative ingredient.
Why not just use a private/shared encoder split or domain masks instead of conditioning? Multi-task Domain Adaptation for Sequence Tagging partitions features into shared and domain-specific regions and reaches 89.0 F1 on Weibo word segmentation without any adversary. Domain separation networks do something similar with orthogonality losses. These preserve dialect-specific information architecturally, which is the same goal as our shared-value restriction. What they do not do is align shared values to each other within the shared subspace. We include a domain-mask tagger and a private/shared tagger with a marginal adversary on the shared part. If either matches us on syntactic features, the case for per-feature conditioning collapses to a case for partitioning.
Why not just self-train on confident target predictions? Pseudo-labeling uses the same predicted distributions we condition on, with no adversary. In a 2k-token regime where tagger predictions are poor, both approaches inherit the same errors. We run entropy-thresholded self-training with the same confidence weighting. We also run an oracle that conditions on gold Egyptian labels for the labeled subset, to measure how much of our gain is limited by prediction quality. If self-training matches us, the honest conclusion is that confident target supervision, not alignment, does the work.
Why not just annotate more Egyptian data? Zalmout and Habash show that adversarial multitask learning at 2k tokens matches a model trained on 16k. Annotation is therefore the currency we report in. For each method, we state the labeled-token count at which the Egyptian-only tagger catches up.
Decisive checks
The idea succeeds if three conditions hold over five seeds. Full-bundle Egyptian accuracy at 2k and 4k tokens exceeds the reproduced marginal-adversary result. Accuracy on dialect-only tokens (negation, future markers) rises where the marginal adversary lowers it relative to pooled training. The probe shows less cross-value nearest-neighbor confusion for case, number, and mood. It fails, and we report it as failed, in any of three cases: the unrestricted per-feature ablation matches the full method, marginal-plus-uniform matches it on dialect-only tokens, or self-training matches it overall. The UD pairs test whether any surviving gain is a property of dialect adaptation or of morphological alignment in general.
References
- Adversarial Multitask Learning for Joint Multi-Feature and Multi-Dialect Morphological Modeling
- Conditional Adversarial Domain Adaptation
- Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation
- Disentangling Masked Autoencoders for Unsupervised Domain Generalization
- Adaptive Risk Minimization: Learning to Adapt to Domain Shift
- Adversarial Adaptation of Synthetic or Stale Data
- A Morphological Analyzer for Egyptian Arabic
- Neural Factor Graph Models for Cross-lingual Morphological Tagging
- Marrying Universal Dependencies and Universal Morphology
- Multi-task Domain Adaptation for Sequence Tagging