When a multilingual model is fine-tuned on English labels and then applied to, say, Japanese product reviews, two different things change at once. The Japanese sentences land in a slightly different region of the model's internal representation space; call this a language offset. The Japanese test set may also have a different mix of classes, with more five-star reviews or fewer contradictions; call this label shift. The common repair for the first problem is to move the target-language representations so that their overall average or overall distribution matches English. This repair quietly breaks under the second problem. If the Japanese data really contains more positive reviews, its average embedding really does sit closer to the positive class, and "fixing" the average pushes positive examples toward negative. This note proposes estimating the offset and the new class proportions jointly, from unlabeled target text, at test time. It also proposes measuring how much of the remaining cross-lingual error is a third kind of change that no unlabeled method can repair: the same meaning receiving a different label in a different language or market.
In two sentences: current zero-shot fixes either move representations or reweight class probabilities, and each is biased when the other kind of shift is present. We fit both together with a small mixture model on a frozen fine-tuned encoder. The same analysis tells us how much of the zero-shot gap is left over as genuine, unlabeled-unfixable conditional shift.
Core Claim
Domain-adaptation theory already predicts this failure in general form. Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift proves that forcing source and target features to look alike when class proportions differ puts a hard floor under target error. It fixes this with class-ratio weights estimated by a quadratic program inside adversarial training. That work evaluates only on vision benchmarks, and its weight estimator relies on an unadapted classifier whose confusion pattern is assumed to carry over to the target. In cross-lingual transfer, the language offset is exactly what breaks that assumption. The two problems are entangled, and neither the representation-matching literature nor the prior-correction literature handles both in the zero-shot multilingual setting.
Our claim is that a simple parametric description of the shift is enough to disentangle them. The encoder is fine-tuned on English and then frozen. Let $z$ be the pooled sentence embedding, projected to a low-dimensional subspace. Let $\mu_y$ and $\Sigma$ be the English class means and shared covariance. We model the unlabeled target-language embeddings as
$$ p_T(z) = \sum_{y} \pi_y \, \mathcal{N}(z;\ \mu_y + \delta,\ \Sigma), $$
where $\pi_y$ is the unknown target class proportion and $\delta$ is a single class-independent language offset. This is a location mixture, so $\delta$ and $\pi$ are jointly identifiable. Expectation-maximization then has closed-form steps. First, compute each example's class responsibilities. Next, set $\pi$ to the mean responsibility. Finally, set $\delta$ to the responsibility-weighted mean of $z - \mu_y$.
At prediction time we keep the fine-tuned classification head. We feed it $z - \delta$ and multiply its output probabilities by $\pi_y / \pi^{S}_y$, the ratio of target to source class proportions, before renormalizing. The failure of naive centering follows directly from the model. Subtracting the target mean estimates the offset as $\delta + \sum_y (\pi_y - \pi^S_y)\mu_y$, so the error grows linearly with the prior shift. This is the cross-lingual form of the alignment paradox, and it is a testable prediction.
The second half of the claim concerns diagnosis. With a small labeled target sample used only for analysis, we can compute the oracle offset and the oracle proportions. The accuracy that remains missing after applying both oracles is the part of the gap caused by genuine change in $P(Y\mid Z)$. Examples include cultural differences in how star ratings are assigned, or lexical cues that predict the label in English but not in the target language. Nobody has reported this decomposition for multilingual encoders.
Relation to Prior Work
Two lines of work motivate the proposal. The Multilingual Amazon Reviews Corpus builds a six-language, five-star review benchmark and reports a large zero-shot gap: English-trained mBERT reaches a mean absolute error of 93.2 on Chinese against 57.7 when trained on Chinese. Zero-Shot Cross-Lingual Sentiment Classification under Distribution Shift: an Exploratory Study shows that combined language and domain shift sharply degrades mBERT, XLM-R and LaBSE. That study repairs the damage only by rewriting inputs with GPT-3.5 or adding human counterfactual edits. It never models label proportions or representation shift, and it covers only binary sentiment. We work in representation space, handle multi-class and ordinal labels, and treat label shift explicitly.
Other work supplies the machinery. Deep Transfer Learning with Joint Adaptation Networks aligns joint distributions of features and predictions during training. Joint distribution optimal transportation for domain adaptation does the same with optimal transport couplings, at quadratic cost in dataset size. Improving Cross-Lingual Sentiment Analysis via Conditional Language Adversarial Nets brings conditional adversarial alignment to cross-lingual sentiment, but on an LSTM trained from scratch, four languages, and an assumed fixed class prior. All of these retrain the network with an alignment loss. We instead keep the fine-tuned encoder fixed and fit a handful of parameters per target language. That makes the method cheap to run for every new language and market.
Optimal Transport Posterior Alignment for Cross-lingual Semantic Parsing is the strongest representation-alignment result in the file, but it needs 1–10% labeled target data. We are strictly zero-shot for the method itself. Two further papers constrain the design. Domain Adaptation via Transfer Component Analysis established that matching feature marginals is insufficient when conditionals differ, which is why marginal centering is our foil rather than our method. Mitigating Both Covariate and Conditional Shift for Domain Generalization found that normalizing features can hurt when pretrained geometry is already aligned. This warns that forcing an offset where none exists could cost accuracy, so we test whether the fitted $\delta$ shrinks to near zero for close languages.
Research Program
We fine-tune XLM-R (Unsupervised Cross-lingual Representation Learning at Scale) and mBERT on English data only, for each task. We then extract pooled embeddings and project them to a subspace built from English data alone. That subspace combines the discriminant directions between classes with the top principal components, and its dimension is chosen on English validation data. From the English training set we fit class means and a shared covariance.
For each target language we run the joint expectation-maximization procedure on unlabeled target test inputs, following the transductive protocol common in domain adaptation. We also report a variant fit on a disjoint unlabeled target pool, so that fitting and evaluation use different examples.
Three extensions test where the simple model breaks. The first allows a low-rank class-dependent offset, which captures languages where classes move by different amounts. The second replaces the Gaussian responsibilities with the classifier head's own posteriors, testing whether the Gaussian assumption matters. The third fits separate offsets per layer, checking whether the top layer is the right place to intervene.
The diagnostic analysis uses a small labeled target sample, never used for fitting. From it we compute the oracle offset (target minus source class-conditional means) and the oracle proportions. Comparing the fitted offset and proportions against these oracles, and measuring the accuracy that remains after applying both oracles, splits the zero-shot gap into three parts. One part is removable by fixing the offset, one by fixing the proportions, and the rest is residual conditional shift. We report this split per language and per task.
Baselines
The naive reference is plain zero-shot transfer: the English-fine-tuned encoder applied unchanged. For XNLI the file reports XLM-R at 80.9% average accuracy on the balanced test sets (Unsupervised Cross-lingual Representation Learning at Scale). We reproduce this number with our own fine-tuning before any comparison. For MARC, the reported reference is mBERT's English-to-Chinese mean absolute error of 93.2 (The Multilingual Amazon Reviews Corpus). Beating the naive reference on the balanced sets tests the offset term alone. Beating it on the skewed sets tests the full method.
The two single-fix baselines isolate the key ingredient, and they are the ablations that matter most. The first, offset-only, applies per-language mean centering, or equivalently our model with $\pi$ fixed to the source proportions. The second, prior-only, runs classic expectation-maximization prior re-estimation on the classifier's outputs without moving representations, or equivalently our model with $\delta = 0$. A sequential variant, centering first and then prior correction, tests whether estimating the two jointly matters or whether ordering the steps is enough. All of these share the same frozen encoder and projection, and none is tuned on target labels. The claim requires the joint method to beat all three under skew and to match the offset-only baseline on balanced data.
The strongest existing training-time competitor is importance-weighted conditional adversarial adaptation. We port IWDAN and IWJAN from Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift to XLM-R fine-tuning, with a CLAN-style discriminator conditioned on predictions (Improving Cross-Lingual Sentiment Analysis via Conditional Language Adversarial Nets). They receive the same unlabeled target text and the same backbone, and they get a hyperparameter search budget equal to the total compute of our fitting plus English fine-tuning, selected on English validation only. No reported result found on Lacuna for IWDAN or IWJAN on any cross-lingual text benchmark. The only reported gains are on vision, for example 92.54% versus DANN's 83.24% on label-skewed digits. On the language-and-domain benchmark we additionally compare against the GPT-3.5 summarization method of Zero-Shot Cross-Lingual Sentiment Classification under Distribution Shift: an Exploratory Study, rerun on its own binary benchmark. Beating the adversarial methods means equal or better accuracy at a small fraction of their cost. Losing to them while staying close would still make the test-time method the practical choice, and we would report it that way.
Why not just…?
Why not just re-estimate the class proportions with expectation-maximization or confusion-matrix inversion and skip representations entirely? This is the most serious threat. Both techniques assume the classifier's pattern of mistakes given the true class is the same in source and target. A language offset changes that pattern, and that is precisely the chicken-and-egg problem the file lists for importance weighting. Whether the offset is large enough in XLM-R's top layer to matter is an empirical question. The skew sweep answers it directly. If prior-only matches the joint method across languages, and the fitted offset is near zero, we will report that the language offset is negligible and that prior correction suffices. That would itself be a useful finding.
Why not just center each language's embeddings? Centering is the one-line version of marginal alignment, and under label shift our model shows it is biased by $\sum_y(\pi_y-\pi^S_y)\mu_y$. This mirrors the marginal-versus-conditional gap reported for DAN and RevGrad against JAN, at 80.4% and 82.2% versus 84.3% on Office-31 (Deep Transfer Learning with Joint Adaptation Networks). The prediction is sharp: centering's accuracy should fall roughly linearly as skew grows, while the joint method's should stay flat. If it does not, the argument fails.
Why not just translate the test set into English? Translation removes the offset but not the label shift, because the Japanese market's star mix survives translation, so prior correction is still needed. The file also records translation noise, such as inconsistent entities in PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification, and weakness on low-resource languages (XNLI: Evaluating Cross-lingual Sentence Representations). We will run translate-test with and without our prior step, so the comparison shows which shift each method actually removes.
Why not just rewrite inputs with an LLM? Summarization to ten tokens works for binary sentiment, but it discards the premise-hypothesis structure that NLI depends on and the gradations that five-star rating depends on. We test it where it was designed to work, on its own binary benchmark, and treat it as complementary: the rewritten inputs can still be offset- and prior-corrected.
Proposed Experiments
The first experiment is a controlled skew sweep, following the subsampling protocol used for label-skewed digits and VisDA in the generalized label shift paper. Target test sets of XNLI (15 languages) and MARC (six languages, five-star labels) are resampled to class proportions at several Dirichlet concentrations, from balanced to heavily skewed. The metrics are accuracy for XNLI, mean absolute error for MARC, and error in the recovered proportions and offset against the oracles. The decisive check is the interaction between method and skew. Centering should degrade with skew, prior-only should degrade on distant languages, and the joint method should do neither.
The second experiment uses natural shift on the benchmark of Zero-Shot Cross-Lingual Sentiment Classification under Distribution Shift: an Exploratory Study: IMDb to Amazon, Tweets, and Restaurants across 13 languages. Here language, domain and class mix change together, and we compare against the summarization and counterfactual baselines.
The third experiment is the gap decomposition. It reports, per language on MARC and XNLI, how much of the zero-shot gap each oracle removes and how much residual conditional shift remains. We expect typologically distant languages such as Chinese and Japanese to have larger offsets, and five-star rating to show more residual conditional shift than NLI. PAWS-X serves as a stress case: its failure comes from surface-overlap cues, which should appear as residual conditional shift that no offset or prior fix can touch.
The method fails if, on skewed MARC and XNLI, it does not beat the better of prior-only and sequential correction on a majority of languages. It also fails if the fitted offsets and proportions do not track their label-computed oracles. In either case the entanglement argument is wrong for modern encoders, and the decomposition result stands as the contribution.
References
- Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift
- The Multilingual Amazon Reviews Corpus
- Zero-Shot Cross-Lingual Sentiment Classification under Distribution Shift: an Exploratory Study
- Deep Transfer Learning with Joint Adaptation Networks
- Joint distribution optimal transportation for domain adaptation
- Improving Cross-Lingual Sentiment Analysis via Conditional Language Adversarial Nets
- Optimal Transport Posterior Alignment for Cross-lingual Semantic Parsing
- Domain Adaptation via Transfer Component Analysis
- Mitigating Both Covariate and Conditional Shift for Domain Generalization
- Unsupervised Cross-lingual Representation Learning at Scale
- PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification
- XNLI: Evaluating Cross-lingual Sentence Representations