Numbers as a Pivot Language: Zero-Shot Cross-Lingual Transfer for Mixed-Type Tables

A table of customer records, patient visits, or census entries is only partly written in a human language. Column names, category labels such as "self-employed" or "Selbstständig", free-text notes, and the names of the prediction classes change when the table moves from English to German or Hindi. Ages, incomes, and lab values stay the same. Today's tabular foundation models are pretrained networks that make predictions on new tables without per-table architecture changes, and the ones that read text, such as TabSTAR, see the whole table through an English text encoder. When the language changes, they appear to lose everything at once.

We propose to treat the numerical columns as a pivot. The idea is to use their language-independent statistics to work out which foreign-language categories and columns correspond to which English ones, with no translation and no target-language labels, and then feed the corrected representations into an English-trained tabular model. Concretely, the German category whose rows have the same income-and-age profile as the English "self-employed" rows is probably "self-employed". A multilingual text encoder supplies a noisy prior, and the numbers confirm or correct it. The work has two parts. First, a measurement study decomposes the cross-lingual "transfer gap" of a mixed-type tabular model by which part of the table carries language. Second, a transductive alignment method uses the unlabeled target-language rows to close that gap.

Core Claim

Cross-lingual transfer in tables differs from cross-lingual transfer in sentences because the shift is partial. Some features move to a new language and others do not. The multilingual NLP literature documents a persistent gap when moving from English to non-Latin and morphologically rich languages; on MLQA, XLM falls from 68.0 F1 in Spanish to 54.8 in Arabic and 48.8 in Hindi (MLQA). That literature has no analogue of the untranslated numerical columns, which are available in every table.

Our claim is that, for mixed-type tables, most of the cross-lingual gap sits in the representations of categorical values and target-label names rather than in free text. We further claim that a data-driven alignment between target and source categories can recover a substantial share of that gap, especially for languages where both the multilingual encoder and machine translation are weakest. The alignment scores candidate matches by the conditional distributions of numerical columns combined with multilingual embedding similarity. If the first claim is false, and the gap lives mainly in free text, numerical anchoring cannot help, and a multilingual encoder swap is the whole story. The decomposition study settles this before the method is built.

Relation to Prior Work

Two lines of work touch this intersection, and neither studies it directly.

The first line brings multilingualism to tables by flattening them into text. XInfoTabS translates infoboxes into ten languages and evaluates mBERT and XLM-RoBERTa on tabular inference. M3TQA translates tables into 97 languages with an LLM pipeline and fine-tunes Llama-3.1-8B for table question answering. Both motivate our work: they document large gaps, especially for Arabic, and show that translating isolated cells without sentence context produces entity errors. Neither treats numerical columns as distributions, and both rely on translation, whereas our method is translation-free.

The second line models heterogeneous tables but stays in English. TabSTAR represents each numeric value both as a z-score and as a quantile-bin phrase. It unfreezes the top layers of an e5-small-v2 encoder and passes candidate class names as input tokens, and it reaches normalized AUROC 0.83 against 0.69 for tuned CatBoost on its 50-dataset text-bearing benchmark. UniTabE fuses column name, value, and type into per-cell units. CTSyn embeds column names and categories with GTE-large to condition cross-table synthesis. The CTSyn authors report that cryptic headers degrade performance, which constrains our design: any method that depends only on header semantics will fail for exactly the tables where language is a problem. TabSTAR supplies our backbone and English source model. ARISE enriches categorical values with LLM-generated descriptions for clustering, which is the opposite direction from ours: it adds language to categories, while we use non-linguistic statistics to decide what a category means.

Two negative results shape the method. Robust Variational Autoencoders for Outlier Detection in Mixed-Type Data shows that likelihoods on discrete and continuous scales cannot be compared directly. For that reason our data term uses per-column, rank-normalized one-dimensional distances rather than a pooled likelihood. The zero-shot reading comprehension study (Hsu et al.) found that fine-tuning on machine-translated Chinese underperformed pure zero-shot transfer, 78.82 F1 zero-shot against 88.68 for native training. This bears on the unresolved tension the research file identifies between translation-heavy tabular work and translation-free transfer in NLP. Our experiments test that tension directly on tables.

Research Program

Setting. A source table in English comes with labels, and the TabSTAR model is trained on it (and pretrained on TabSTAR's corpus). The target table covers the same or a closely related prediction task. Its column names, categorical values, free text, and class names are in another language, and it arrives with unlabeled rows only. The numerical columns keep their meaning but may shift in scale; for example, a different currency or a different national income distribution. Every numerical column is therefore rank-transformed within its own table, following the quantile treatment in CTSyn and TabSTAR.

Stage one: decomposing the gap. We build target tables in which only one component is in the target language at a time: headers only, categorical values only, free-text fields only, class names only, and then all four together. We measure how much each component contributes to the drop from English performance, per language. This is cheap, and it decides whether the method in stage two is worth building.

Stage two: numerical anchoring. For a categorical column, each target value $v$ gets a signature built from the rank-normalized conditional distribution of every numerical column $j$ given that category. Each source value $u$ gets the same signature. The cost of matching $v$ to $u$ is

$$ C(v,u) = \lambda \sum_j W_1\big(F_j(\cdot \mid v), F_j(\cdot \mid u)\big) + (1-\lambda)\big(1 - \cos(e_v, e_u)\big), $$

where $F_j(\cdot \mid v)$ is the conditional distribution of column $j$ among rows with value $v$, $W_1$ is the one-dimensional Wasserstein distance, and $e_v$, $e_u$ are multilingual encoder embeddings. The weight $\lambda$ trades data evidence against textual similarity. We compute a soft correspondence between target and source values with entropy-regularized optimal transport, using each value's frequency as its mass so that rare categories are not forced into common ones.

The target value then enters the English-trained model with a blended representation: a mixture of its own multilingual embedding and the transport-weighted average of its matched source embeddings. Column names are aligned the same way, using marginal distributions and co-occurrence with already-aligned categories. Class names are aligned through the model's own predicted class distribution on the unlabeled target rows. The same alignment can be passed to CTSyn's schema conditioning, which gives a secondary test on zero-shot synthesis for a non-English schema.

Stage three: when the anchor lies. Anchoring assumes that categories keep roughly the same numerical profile across the two populations. We test this directly with controlled shifts: resampling target rows so that category-conditional distributions drift by known amounts, and tracking where anchoring stops helping. When the data term disagrees strongly with the text term, the method should fall back toward $\lambda = 0$. We select $\lambda$ with an unsupervised criterion, the transport cost at the solution, never with target labels.

Baselines

The naive reference point is the English-trained TabSTAR applied directly to the target table with its original English e5-small-v2 encoder. This shows how badly an English model fails. A second naive point uses only the numerical columns, which shows what the language-bearing features are worth at all.

The strongest competitor on this question is TabSTAR with its encoder replaced by a multilingual encoder of the same size (multilingual E5) and retrained on the same English pretraining corpus. It is the natural answer from the cross-lingual literature: shared multilingual geometry, as in mBERT or XLM-R, with nothing tabular-specific added. We give it the same pretraining data, compute, and tuning budget as our method, and the same unlabeled target rows for any unsupervised adaptation it can use (for example, re-estimating quantile bins). Our method must beat it on target-language AUROC averaged over languages, and must beat it by a clear margin on Arabic and Hindi, where encoder alignment is weakest.

The second strong competitor is translate-test. The target headers, categories, text, and class names are translated into English with a strong MT system, using XInfoTabS-style category conditioning where feasible, and then fed to the English TabSTAR.

For reference, TabSTAR reports normalized AUROC 0.83 on its 50-dataset benchmark, but that figure is English-only. We found no reported result on Lacuna for cross-lingual zero-shot prediction on mixed-type tables, so every non-English number in this study is new.

The ablations isolate the key ingredient: anchoring with only the encoder term ($\lambda = 0$), anchoring with only the data term ($\lambda = 1$, which is language-blind matching), and hard one-to-one versus soft assignment.

Why not just…?

Why not just translate the table into English first? This is the most serious threat, and for German, French, or Spanish tables with ordinary vocabulary it may simply win. The evidence against it is specific. XInfoTabS found that translating cells without sentence context garbles entities and that quality drops sharply for Arabic, and Hsu et al. found that translation noise can hurt more than it helps. Neither result was measured on mixed-type prediction, however. Our experiments therefore report translate-test per language and per component. We also test the combination of translate-test with anchoring as a correction step, since anchoring can repair mistranslated categories. If translate-test matches anchoring everywhere, we will report that the numbers add nothing beyond MT.

Why not just swap in a multilingual encoder? This is the baseline anchoring must beat. Its weakness is predicted by the MLQA results: multilingual geometry is uneven across languages. It also cannot handle strings that carry little meaning, such as codes, abbreviations, and cryptic headers, which the CTSyn authors report as a failure mode. The $\lambda = 0$ ablation is exactly this approach with transductive smoothing, so the comparison isolates the contribution of the data term.

Why not just embed the text with a frozen multilingual model and train CatBoost? In the zero-shot setting there are no target labels, so CatBoost must be trained on English embeddings and applied to target embeddings. That inherits all the encoder-alignment error, and TabSTAR's comparison shows static embeddings trailing a model with unfrozen encoder layers (CatBoost 0.69 against TabSTAR 0.83 in English). We run it anyway as a cheap reference point.

Why not just serialize the table and prompt a multilingual LLM? M3TQA shows this can work for question answering. TabSTAR's comparisons suggest, however, that models reading only text handle numerical magnitudes poorly in prediction tasks. We include a serialized-prompt baseline on the classification datasets, run with a fixed prompt budget.

Proposed Experiments

Benchmarks. We construct the target tables from the text-bearing classification datasets in TabSTAR's 50-dataset benchmark and from Adult. We translate them into the ten XInfoTabS languages with a pipeline modeled on M3TQA's (serialization, cell correction, back-translation). The MT system used to build the benchmark differs from the one used for the translate-test baseline, so that translate-test does not simply invert its own output. A native-speaker-checked subset for Arabic, Hindi, and German measures translation artifacts. Two further controls use the same data: a cryptic-header variant with headers replaced by "Var_k", and the controlled-shift variants from stage three. XInfoTabS, whose infoboxes are mostly written in human languages rather than numbers, serves as an out-of-distribution check where anchoring should help little. The CTSyn synthesis test reports its correlation and authenticity metrics on translated Adult.

Decisive checks. The project rests on four checks. First, the decomposition must show that categorical values and class names account for most of the gap; otherwise we stop before stage two. Second, the data-only ablation must recover part of the gap on its own, which proves the numbers carry alignment signal. Third, the full method must beat both the multilingual encoder swap and translate-test on the non-Latin languages at matched compute. Fourth, the gain must shrink in a predictable way as the controlled distribution shift grows. A method that survives all four gives a clear answer to the translation-versus-transfer tension for tables: when the table's own numbers are used as the dictionary, explicit translation stops being necessary.