Search engines usually rank in two steps. A cheap retriever pulls about a hundred candidate documents, and then a slower "re-ranker" model reorders them by reading each query and document together. Recent work shows that a small re-ranker of about 3 billion parameters ranks much better when it first writes a short explanation of why a document is or is not relevant. That explanation is learned by imitating a much larger "teacher" model, a technique called reasoning distillation. The catch is cost: writing an explanation for every one of a hundred candidates is slow. In cross-lingual search, where an English query is matched against, say, Hausa or Somali passages, the cost grows further, because the usual fix for weak non-English understanding is to machine-translate every passage into English first.
We propose a compact cross-lingual re-ranker that writes its explanation only for the candidates it is genuinely unsure about. It writes that explanation in English, so the explanation also serves as a short, query-focused translation of the passage. The key ingredient is a gate that decides which candidates get an explanation. That gate depends on the model's confidence, and multilingual models are known to become badly overconfident in distant languages. A gate tuned in English will therefore skip reasoning exactly where reasoning is most needed. Our claim is that 10–100 judged examples in the target language are enough to recalibrate the gate per language. With that recalibration, the gated re-ranker matches the quality of reasoning about every candidate at a fraction of the generated tokens, and it beats the translate-then-rank pipeline once translation cost is counted.
Core Claim
Three findings in the literature, taken together, motivate the design. First, explanations matter for small re-rankers. InteRank (Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking) distills relevance explanations from Llama 3.3 70B into Llama 3.2 3B. Removing the explanation step drops nDCG@10 on the reasoning-heavy BRIGHT benchmark from 21.5% to 14.4%. Second, explanations are expensive. DeAR (DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation) separates a fast scorer that writes no reasoning (2.2 seconds per query on TREC DL19) from a slower listwise reasoning stage (11.16 seconds), and the reasoning stage only fits the top 20 candidates. Third, confidence is unreliable across languages. On the Calibration of Massively Multilingual Language Models reports that XLM-R's calibration error (ECE, the gap between stated confidence and actual accuracy) on XNLI rises from 7.32 in English to 19.07 in Swahili. Few-shot target-language data brings Swahili down to 5.69, compared with 11.54 when calibrating on English data.
The new idea is to treat the few-shot target data as a calibration budget for a compute-allocation gate. We do not use it as ordinary task fine-tuning.
The student first produces a fast score $z$ by reading the logit of a relevance token, with no generated text. For a candidate in language $\ell$, the student writes an English rationale and rescores only when
$$\left|\sigma(z / T_\ell) - 0.5\right| < \tau_\ell .$$
Here $\sigma$ is the logistic function, $T_\ell$ is a temperature for language $\ell$, and $\tau_\ell$ is a band width for language $\ell$. Both are fit on the $k$ judged target-language examples so that the expected fraction of candidates sent to reasoning meets a fixed token budget. We predict that an English-fit $(T, \tau)$ will send too few Hausa or Somali candidates to reasoning, because overconfidence pushes their scores out of the uncertain band. Per-language fitting should close most of that gap.
Research Program
Training the student. We train one Llama 3.2 3B student, matching InteRank's scale, with two heads that share weights: a direct-score mode and an explain-then-score mode. Following InteRank, the teacher is Llama 3.3 70B. Training pairs come from the MIRACL training languages (Making a MIRACL), with queries in English and in the native language paired against native passages. For each pair the teacher writes an English rationale that names the query-relevant content of the passage, followed by a graded score. Following the "solution-in-prompt" idea of Teaching Small Language Models to Reason, the teacher sees the gold label so that its rationales are correct, and we keep only rationales whose score agrees with the label, as in Specializing Smaller Language Models towards Multi-Step Reasoning.
Why the rationale is in English. We write rationales in English on purpose, which puts us on one side of a real tension in the literature. XCOT (XCOT) and Representational Isomorphism and Alignment of Multilingual Large Language Models treat English as the shared coordinate frame. Understanding and Mitigating Language Confusion in LLMs shows models drifting into English against the user's wishes. For ranking, the rationale is never shown to a user, so drifting into English costs nothing. We still test rationales written in the passage language as an ablation.
Writing the rationale in English is also our answer to the translation pipeline. Zero-Shot Cross-Lingual Reranking with Large Language Models for Low-Resource Languages finds that RankGPT4 ranks better with its own query translations than with Google's (0.3835 vs 0.3602 nDCG@20 on CIRAL). That paper evaluates only and trains nothing. The result suggests a model's own English rendering suits it better than external translation. Our rationale is that kind of self-rendering, restricted to the part of the passage that matters for the query.
Few-shot adaptation. For each CIRAL target language (Hausa, Somali, Swahili, Yoruba) we draw $k \in \{0, 5, 15, 50, 100\}$ judged query–passage pairs. We compare two uses of this budget. The first fits only the gate parameters $(T_\ell, \tau_\ell)$. The second fits the gate and also a small LoRA language adapter (a low-rank add-on to the model's weights), in the spirit of the separate language and ranking adapters of Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval. We include $k = 15$ because Cross-lingual Few-Shot Learning on Unseen Languages finds that weight updates overtake in-context learning at around 15 examples. We expect a similar crossover between gate-only and adapter adaptation.
Measuring cost. Cost is measured as generated plus input tokens per query and wall-clock time per query on a single A100. For pipelines that use translation, the cost of translating the candidate pool is included.
Baselines
The naive reference point is the first-stage ranking with no re-ranking: BM25 or mDPR candidates in their original order. Alongside it we run the student's direct-score mode with no reasoning at all. This shows what the cheapest version of our own model achieves, and every reasoning gain is measured against it.
The strongest quality competitor is the student in always-reason mode, writing a rationale for all top-100 candidates. This is InteRank's recipe applied cross-lingually. InteRank's reported English figure is 27.4% average nDCG@10 on BRIGHT. We found no reported result on Lacuna for any distilled reasoning re-ranker on CIRAL or MIRACL. Always-reason defines the ceiling we aim to approach at lower cost. Beating it means coming within 1 nDCG@20 point while generating at most 40% of its rationale tokens.
The strongest existing system on CIRAL is RankGPT4 listwise re-ranking with self-translated queries, at 0.3835 average nDCG@20 (Adeyemi et al.). We treat that number as an upper reference rather than a target, since it uses a far larger proprietary model. Coming within a few points of it at 3B scale would itself be a result.
The translate-then-rank pipeline machine-translates every candidate passage into English with an open translation model and then applies the same student in English. It receives the same training data, the same 100 candidates, and the same tuning budget, and its translation time is charged to its cost. Adeyemi et al. report that this strategy gains up to 7 nDCG@20 over direct cross-lingual ranking, and the CIRAL overview (Overview of the CIRAL Track at FIRE 2023) finds document translation best in three of four languages. That makes it the strongest cheap competitor.
The non-reasoning cross-encoder baseline is the adapter-based reranker of Litschko et al., trained on the same data and given the same $k$ target examples for its language adapter.
The rank-based cascade reasons only over the top-$m$ candidates by fast score. This is the DeAR-style design, with $m$ chosen so its token budget matches ours exactly.
The key ablation keeps everything fixed except the gate. We compare a gate fit on English data, a gate fit with zero-shot temperature scaling, and a gate fit per language on $k$ target examples. If per-language fitting does not beat the English-fit gate at equal budget, the central claim fails.
All systems share the same first-stage candidates, the same teacher data, and the same hyperparameter search budget. Each is tuned on MIRACL development languages that are disjoint from the test languages.
Why not just…?
Why not just translate everything and use a fast English re-ranker? This is the most serious threat, and we do not dismiss it. On CIRAL the passages are short news windows of at most six sentences, so translation may be cheap enough to make our gain on cost negligible. The evidence against it is indirect. External translation underperforms the model's own rendering (0.3602 vs 0.3835 nDCG@20 for RankGPT4), and translation coverage is weak for the lowest-resource languages. The translate-then-rank baseline, with translation cost charged, settles the question. If it matches us on both quality and cost, we report that the English-rationale idea adds nothing beyond translation.
Why not just a cross-encoder with no reasoning? For reasoning-heavy English retrieval, InteRank reports that MiniLM cross-encoders score below 20% nDCG@10 on BRIGHT. CIRAL and MIRACL queries, however, are mostly factoid, and reasoning may help little there. Our design hedges against this. If reasoning is rarely useful, the calibrated gate should send few candidates to reasoning, and the system's cost should approach that of the cross-encoder. We will report the fraction of candidates sent to reasoning per language and per dataset. A near-zero fraction with no quality gain would show that a plain cross-encoder suffices for these benchmarks.
Why not just reason over the top-$m$ candidates? A rank cascade is simpler and needs no calibration. Its weakness is structural: it spends reasoning on confident top-ranked candidates and never reaches uncertain candidates ranked lower. Overconfidence in distant languages makes the fast ordering itself less trustworthy. The equal-budget comparison against the rank cascade tests this directly, and we expect its advantage over our gate to shrink as a language gets closer to English.
Why not just calibrate once on English, or prompt a large model with a few examples? The calibration paper's zero-shot temperature scaling plus label smoothing already cuts XNLI ECE from 13.34 to 4.89, so target data may be unnecessary. The gate ablation tests this. Prompting a large model with in-context examples avoids training, but it pays the latency that motivates this work: RankZephyr takes 21.58 seconds per query on TREC DL19 (DeAR). In-context learning is also unstable in unseen languages, at about 47–50 F1 versus 64.92 for fine-tuning on NusaX (Winata et al.).
Proposed Experiments
The main test uses CIRAL for low-resource cross-lingual ranking and MIRACL for monolingual ranking in 18 languages. The MIRACL hybrid BM25+mDPR first stage reaches 0.578 nDCG@10 and 0.889 Recall@100, so re-ranking gains there are not limited by recall. We report nDCG@20 on CIRAL, following its judging depth, and nDCG@10 on MIRACL. We plot quality against tokens per query for each system, sweeping the reasoning budget from 0% to 100% of candidates. Separately, we report ECE of the fast score per language before and after gate fitting, and we correlate the reduction in ECE with each language's syntactic distance from English, using the URIEL distances adopted in the calibration paper.
Three checks decide the outcome. The first: at a 30–40% reasoning budget, the per-language gate must sit above both the English-fit gate and the rank cascade on the quality–cost curve for Hausa, Somali and Yoruba, and it must reach within 1 point of always-reason. The second: the gate's advantage must grow with distance from English. A flat advantage would mean the gain does not come from calibration. The third: against translate-then-rank with translation cost included, we must win on cost at equal quality in at least three of the four CIRAL languages. We also report the $k$ at which adding a LoRA adapter starts to beat gate-only adaptation, and whether rationales written in the passage language perform worse than English rationales, which bears on the English-anchor question.
References
- Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking
- DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation
- On the Calibration of Massively Multilingual Language Models
- Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages
- Teaching Small Language Models to Reason
- Specializing Smaller Language Models towards Multi-Step Reasoning
- XCOT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning
- Representational Isomorphism and Alignment of Multilingual Large Language Models
- Understanding and Mitigating Language Confusion in LLMs
- Zero-Shot Cross-Lingual Reranking with Large Language Models for Low-Resource Languages
- Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval
- Cross-lingual Few-Shot Learning on Unseen Languages
- Overview of the CIRAL Track at FIRE 2023: Cross-lingual Information Retrieval for African Languages