Prior-Aware Classifier-as-Critic Alignment in Frozen Multilingual Models

A multilingual language model fine-tuned on English task labels can often do the same task in other languages with no labeled data in those languages. This is called zero-shot cross-lingual transfer. It works because pretraining places sentences with the same meaning near each other across languages. It degrades badly in two situations: typologically distant languages, where English-to-Hindi extractive QA falls to 48.8 F1 against 68.0 for Spanish (MLQA), and target populations whose label mix differs from English, for example far more positive reviews. Domain adaptation offers a fix: during training, also show the model unlabeled target-language text and push the target data toward the region where the English classifier works well. The adaptation literature also argues that this alignment should be trained jointly with the task loss rather than as a separate stage, because alignment that ignores the labels can merge distinct classes.

We propose doing this alignment inside a backbone that is never modified. All adaptation goes through prefix vectors, which are small learned key-value vectors prepended to every attention layer. The task classifier itself serves as the alignment critic, so there is no separate domain discriminator. The new ingredient is a correction for label-mix (prior) shift. The best discriminator-free alignment loss works by rewarding diverse predictions on target data, and when the target language's label distribution is skewed, that reward pushes predictions in the wrong direction. We estimate the target label distribution online from the model's own predictions and resample English batches to match it. Everything happens in a single training stage.

Core Claim

The claim has three parts, tested together but separable by ablation. First, a frozen-backbone, prefix-only model trained with joint alignment against unlabeled target-language inputs beats English-only prefix tuning. That baseline already beats LoRA on XNLI, 78.7% vs 74.4% average accuracy (Zero-Shot Cross-Lingual Transfer using Prefix-Based Adaptation). Second, the alignment signal should come from the task classifier. Third, under label-mix shift, uncorrected classifier-as-critic alignment is worse than no alignment, and the online prior correction removes that harm.

The third part is the sharpest prediction and the one that distinguishes this proposal from simply transplanting a vision technique into NLP.

Relation to Prior Work

Three papers already treat cross-lingual transfer as unsupervised domain adaptation, and each leaves a specific door open.

Unsupervised Domain Adaptation for Cross-lingual Text Labeling trains two classifiers to disagree on target text while fine-tuning all of mBERT end-to-end, plus a local-smoothness regularizer. It reaches 77.9% on Italian MLDoc, against a 60.6% baseline. It updates the whole backbone and assumes equal label mixes. We freeze the backbone and model the prior explicitly.

Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model splits features into domain-invariant and domain-specific parts using only English unlabeled text. It improves German book reviews from 80.4% to 89.2%. Because it never looks at target-language text, it cannot correct drift specific to that language. Our method aligns directly against target-language inputs.

Cross-Lingual Transfer with Class-Weighted Language-Invariant Representations motivates the proposal most directly. It shows that label-mix shift breaks naive alignment, and that conditional feature invariance governs transfer. Its fix is importance-weighted adversarial alignment with a separate Wasserstein critic, followed by a separate distillation stage, on fully fine-tuned encoders. We keep its weighting idea but change three things: we remove the critic network, merge the two stages into one, and restrict all trainable parameters to prefixes and a small head.

The method itself comes from Discriminator-free Adversarial Domain Adaptation (DALN). DALN reuses the softmax classifier as a critic through a nuclear-norm discrepancy, which rewards target predictions that are both confident and diverse across classes. It reports 71.8% on Office-Home vs 57.6% for DANN.

DALN's diversity reward also constrains our design. Diversity is the right inductive bias when target classes are roughly balanced. It is the wrong one when, say, 90% of target reviews are positive. Our weighting is aimed exactly at this weakness, which the file's gap analysis does not identify.

The prefix paper constrains the design in a different way. Its evidence that modifying weights erodes multilingual ability is why the alignment gradient is allowed to reach only the prefixes. The sequential-versus-joint argument comes from Information-Theoretical Learning of Discriminative Clusters and The Necessity of Combining Adaptation Methods. Both show that alignment decoupled from the task loses class boundaries. That argument is why we refuse a two-stage pipeline, though we test that choice rather than assume it.

Research Program

The model is a decoder LLM of the kind used in the prefix paper (Llama or Mistral class), with all weights frozen. The trainable parts are prefix vectors $\theta$ and a linear classifier $h$ that reads the final hidden state at the answer position and outputs $K$ class probabilities. The labels are three for natural language inference and two to five for reviews, so $K$ is tiny.

Each step draws an English labeled batch and an unlabeled target-language batch of size $B$. The step stacks the classifier's softmax outputs into matrices $P_S$ and $P_T$, each of size $B \times K$.

The target prior estimate $\hat\pi_T$ is a running average of target soft predictions. It is refined with the standard expectation-maximization correction for label shift. This combination is a routine choice, and we test it separately against using raw averages. English examples of class $y$ are sampled with probability proportional to $\hat\pi_T(y)/\pi_S(y)$, where $\pi_S$ is the known English label distribution. The source batch therefore carries the target label mix.

Training then solves

$$\min_{\theta}\max_{h'}\; \mathcal{L}_{CE}(h;\theta) \;+\; \lambda\Big(\tfrac{1}{B}\|P_S\|_* - \tfrac{1}{B}\|P_T\|_*\Big),$$

with the terms defined as follows. $\mathcal{L}_{CE}$ is cross-entropy on the resampled English batch. $\|\cdot\|_*$ is the nuclear norm, the sum of singular values. $\lambda$ weights the alignment term. $h'$ is the classifier as seen through a gradient-reversal layer, following DALN.

The classifier tries to make the source predictions look more confident and diverse than the target predictions. The prefixes try to close that gap. Because the source batch has been resampled to the estimated target label mix, the prefixes are no longer rewarded for spreading target predictions uniformly. With $K \le 5$, the singular value decomposition costs essentially nothing per step. This removes the computational objection the file raises against nuclear-norm methods.

The unlabeled target text is drawn from the training or held-out splits of each benchmark with labels discarded. It is never drawn from the test split.

Baselines

The naive reference point is English-only prefix tuning evaluated zero-shot. For context, the file's prefix-paper figure on XNLI, 78.7% average accuracy (Zero-Shot Cross-Lingual Transfer using Prefix-Based Adaptation), was obtained with the same frozen-backbone recipe. This is the strongest competitor among methods that use no target data, and it shares our parameter budget exactly. We rerun it with our backbone, prefix length, and tuning budget rather than relying on the reported number. The proposal must beat our rerun on average and especially on distant languages such as Hindi, Swahili, and Urdu.

The strongest existing method that uses target unlabeled data is importance-weighted alignment followed by distillation (IWDA+KD). The file reports its 51.53% average reduction in conditional feature shift on MARC, but no reported result was found on Lacuna for IWDA+KD task accuracy on MARC or XNLI. We therefore reimplement it in two forms. One is faithful: a fully fine-tuned encoder with a Wasserstein critic. The other is matched: the same frozen backbone and prefixes, with the critic network added and two stages. Both get the same unlabeled data and the same number of hyperparameter trials. Beating the matched version isolates our contribution. The faithful version tells us whether freezing costs anything.

For MLDoc we also rerun the two-classifier disagreement method (MDD+VAT) as published, reported at 77.9% Italian accuracy, and in a prefix-only variant.

The ablations isolate each ingredient. Plain DALN-style alignment without prior resampling tests whether the correction matters. Resampling English batches by the estimated prior with no alignment term tests whether alignment matters beyond reweighting. A separate small domain discriminator in place of the classifier-as-critic tests the discriminator-free choice. Running alignment first and then training the classifier on frozen prefixes tests joint versus sequential training. The same objective trained through LoRA instead of prefixes tests whether the frozen-backbone constraint is load-bearing.

Why not just…?

Why not just prefix-tune on English and ship it? This is the default, and it is strong. Its failure mode is the one the proposal targets: no mechanism notices that the target language's inputs or label mix differ. The MLQA gaps on Arabic and Hindi, and IWDA's demonstration that zero-shot transfer degrades under label-mix shift, suggest real headroom. If our method gains under one point on distant languages and under shift, the honest conclusion is that frozen multilingual geometry already does the work. The experiments are designed to make that outcome visible.

Why not just self-train on pseudo-labeled target text? This is the most serious threat. IWDA's own best system includes distillation on target pseudo-labels, and self-training also uses unlabeled target data cheaply. We run prefix-only self-training with confidence thresholding, plus prior-matched pseudo-label sampling, under the same compute. Self-training reinforces confident errors and is known to amplify majority classes under label shift. Our objective instead compares the structure of whole prediction batches against an English reference. If self-training matches us everywhere, the contribution reduces to the prior estimator, and we will report it that way.

Why not just correct the output probabilities for the target prior at test time? Post-hoc reweighting of the classifier's logits by an estimated prior fixes the label mix without any training. If label mix were the only problem, it would match us. It cannot repair misplaced class-conditional features, which IWDA identifies as the actual driver of transfer. The "resampling only" and "post-hoc correction" baselines, compared against the full method on balanced XNLI, separate these effects. Gains on balanced data can only come from alignment.

Why not just translate the test set into English? XNLI reports translate-test in roughly the 65–73% range with older encoders, and notes the inference cost. We include translate-test with an off-the-shelf translation system as a reference on XNLI and MLDoc. We expect it to remain competitive on high-resource languages, and we do not claim otherwise there.

Proposed Experiments

XNLI tests balanced-prior transfer across 15 languages. It answers the question: does classifier-as-critic alignment help when label mixes match?

MARC tests label-mix shift. We follow IWDA's protocol of subsampling target test and unlabeled sets to many controlled label distributions, and sweep the shift from none to extreme.

MLDoc gives a direct comparison with the disagreement-classifier method on document classification.

Metrics are accuracy per language and averaged. On MARC we also report conditional feature shift, measured as IWDA does, so that representation-level change can be linked to accuracy.

The decisive check is the MARC shift sweep, comparing three systems: plain classifier-as-critic alignment, prior-corrected alignment, and the English-only prefix baseline. The proposal's central prediction is a crossover. Plain alignment should help at low shift and fall below the baseline at high shift, because its diversity reward fights the skew. The corrected version should stay at or above the baseline throughout.

If plain alignment never falls below the baseline, the diversity-bias argument is wrong and the weighting is merely a tuning detail. If the corrected version does not beat post-hoc prior correction plus self-training at matched compute, the joint objective is not earning its complexity.

A secondary check compares LoRA and prefixes under the identical objective on distant languages. It tests whether the frozen-backbone constraint matters for alignment, as the prefix paper's results suggest, and not only for plain English fine-tuning.