When two languages share almost no parallel text, such as Lao and Armenian, a common workaround is to translate through a better-resourced "pivot" language in between. A recent refinement hands a large language model several rough drafts and asks it to act as an editor. The obvious next move is to give that editor drafts produced through several different pivot languages, on the theory that independent routes let their mistakes cancel out. The evidence says this can backfire. When pivot routes run through the same underlying multilingual model, their hallucinations are "sticky": the routes make the same mistakes. Averaging them then reinforces the error instead of cancelling it. Investigating Multi-Pivot Ensembling with Massively Multilingual Machine Translation Models reports 18.8% hallucinations for its best multi-pivot ensemble against 7.3% for a single English pivot on M2M100.
Composite materials science has a precise vocabulary for this pattern, and we borrow it as a measurement lens, not as a physical model. Adding reinforcing fibers or particles helps up to a critical loading. Past that point the additions clump into agglomerates that behave like one large defect, and strength falls. Reinforcement helps when it is dispersed and hurts when it is aggregated, whatever the nominal amount. We propose that pivot drafts behave the same way. An LLM editor's quality should be governed by how dispersed the drafts' errors are, not by how many drafts it sees. A cheap, reference-free estimate of error clumping should therefore predict both which pivot drafts to add and when to stop.
Core Claim
The claim has two parts. First, as pivot drafts are added to an LLM editor's context in arbitrary order, quality should follow an inverted-U "loading curve" and hallucinations should rise past a critical count. That critical count should shift predictably with the measured overlap of the drafts' errors. Second, a greedy rule that admits drafts only while their estimated shared-error ratio stays low should match the best fixed number of drafts chosen with hindsight on the test set. It should also beat the single-English-pivot editor on non-English target directions, without needing a reference-annotated development set per language pair. The second point is the problem Investigating Multi-Pivot Ensembling explicitly leaves open.
The overlap measure is defined as follows. For each draft $i$, let $E_i$ be the set of target spans that a source-conditioned checker flags as unsupported by the source sentence. The checker marks target content words with no cross-lingual alignment to the source. Spans in two drafts are matched when their lemmas or embeddings agree. The shared-error ratio between drafts $i$ and $j$ is
$$\rho_{ij} = \frac{|E_i \cap E_j|}{\min(|E_i|, |E_j|)},$$
the fraction of the smaller draft's unsupported content that the other draft repeats. A candidate draft is admitted to the context only if its maximum $\rho$ against already-admitted drafts is below a threshold $\tau$. This threshold is the analogue of a critical loading and is tuned once across language pairs, not per pair.
Relation to Prior Work
The method builds on SCALE: Synergized Collaboration of Asymmetric Language Translation Engines. SCALE lets a frozen LLM edit a single draft from a small specialized translation model, and in its pivot variant it translates the source into English first. SCALE-pivot beat few-shot GPT-4 by an average of 6 COMET points on eight non-English directions. SCALE uses exactly one draft, and its authors note that longer prompts inflate latency. We study the many-draft regime SCALE does not test, and we treat prompt length as a cost the admission rule must justify.
Three lines of work motivate the proposal by providing evidence that multiple routes can help. Simultaneous Multi-Pivot Neural Machine Translation gained 2.4 BLEU from French plus Spanish pivots over French alone, but it needed fully N-way parallel UN data to train a multi-source decoder. We need no such data because the combiner is a frozen LLM. The triangulation work of Cohn and Lapata and Wu and Wang merged phrase tables under the assumption that source and target are conditionally independent given the pivot. Sticky hallucinations are a direct violation of that independence assumption, and $\rho$ is an attempt to measure the violation per sentence. Dialectal and related-language pivoting, through Modern Standard Arabic in Habash et al. and through Slavic neighbours in Tiedemann and Nakov, suggests that non-English pivots contribute information English does not. This is why our candidate pool is built to include them.
One paper constrains the design. Investigating Multi-Pivot Ensembling showed that token-level averaging amplifies shared errors and that even confidence-based selection (MaxEns) trails a single English pivot on hallucinations. We take this as a negative result that rules out selecting drafts by model confidence. Confidence is high precisely where sticky errors are shared, so we select by error overlap instead.
From materials science we reuse only a measurement habit, not any physical law. Studies such as Cui et al. find an optimum at 0.2 wt% nucleating agent, with agglomeration appearing at 0.3 wt%. Wang et al. report porosity rising from 3.85% to 6.86% and bending strength falling from 662 to 530 MPa as TiC rises from 25 to 41 wt%. Takahara et al. attribute fatigue gains to molecularly dispersed rather than aggregated clay. These studies sweep the loading, image the dispersion state, and relate failures to aggregation, not to amount. We apply the same protocol to drafts: sweep the draft count, measure $\rho$, and relate hallucinations to shared errors.
Research Program
We build a candidate pool for each source sentence. It contains a direct NLLB-200 draft from No Language Left Behind and target-language drafts routed through four to six pivots. The pivots are English, a typologically or geographically related language (for example Thai for Lao, or Russian for Armenian), and one or two further high-resource languages. We also add an M2M100 route so that the pool mixes drafts from the same model with drafts from a different model. A frozen open-weight LLM receives SCALE-style triplet demonstrations extended to multiple drafts, in the form of source, several drafts, and gold target, and writes the final translation.
The first study is the loading curve. We add drafts in random order for $K$ from 0 to 6, repeat over several random orderings, and record COMET, spBLEU, hallucination rate, prompt tokens and wall-clock latency. In parallel we record reference-based pairwise error correlation, which lets us validate $\rho$ against the ground truth it approximates.
The second study is a controlled agglomeration experiment, the analogue of varying weight percentage in a lab. We take a clean draft and inject the same synthetic content error, such as a wrong entity, number or negation, into $m$ of the $K$ drafts. We then measure how often the editor reproduces the error as $m/K$ varies. If the editor follows a majority, reproduction should jump near $m/K = 0.5$. That would directly demonstrate the clumping mechanism, independently of natural pivot noise.
The third study is the admission rule. We select $\tau$ on a held-out subset of directions and apply it unchanged to the remaining directions.
Baselines
The naive reference point is the direct NLLB-200 draft with no editor. A second reference is the LLM editor with no drafts at all, few-shot only. Together these show how much any draft-based editing adds.
The strongest competitor for quality is SCALE-pivot with a single English draft, as in SCALE. We reimplement it with the same LLM, the same demonstrations, the same decoding settings and the same NLLB checkpoint, so that the only difference is the draft set. The file gives SCALE's gain of 6 COMET points over few-shot GPT-4 on non-English pivot directions. It gives no absolute COMET value for SCALE-pivot, so we will report our own reimplementation's score as the bar to beat. Beating SCALE-pivot means a statistically significant COMET gain across held-out directions with a hallucination rate no higher.
The strongest competitor on hallucination is a single English pivot without an editor. Investigating Multi-Pivot Ensembling reports this at 7.3% hallucinations on M2M100, against 18.8% for MaxEns. We rerun both English pivoting and MaxEns on the same FLORES-101 directions and the same models, using that paper's hallucination criterion. Our multi-draft system must not exceed the English-pivot rate while improving quality.
The key ablation feeds the editor the same number of drafts chosen by admission rule, but chosen three different ways: at random, by model confidence, and by $\rho$. This isolates whether dispersion, rather than draft count or confidence, carries the effect. An oracle that picks the best $K$ per direction with test references gives an upper bound.
Why not just…?
Why not just use a single English pivot, since it had the fewest hallucinations? This is a real threat. On FLORES-101 Indo-European directions with M2M100 it may simply win. Our experiments settle this per direction. We expect English to suffice when English-centric training dominates, and related-language pivots to add value where the dialectal pivoting of Habash et al. suggests English discards information, as in Southeast Asian and Caucasian pairs. If the admission rule rarely admits anything beyond English, we will report that as the finding.
Why not just generate all pivot drafts and rerank them with a quality-estimation model or minimum-Bayes-risk selection? Minimum-Bayes-risk selection picks the candidate that agrees most with the other candidates. This is the most plausible simpler alternative, and we run it on the identical candidate pool. Its weakness is structural. It rewards agreement, and sticky hallucinations are agreement, so it should fail exactly where $\rho$ is high. The controlled injection study tests this directly: if reranking resists shared injected errors as well as the admission rule does, the editor adds nothing. In that case we would recast the contribution as $\rho$-filtering of the reranker's candidate pool.
Why not just pick the top few pivots per language pair on a development set? This requires references for every pair, which truly low-resource pairs lack. It also chooses a fixed set, whereas error overlap varies by sentence. We include it as a baseline with dev references and ask whether the reference-free rule matches it.
Why not just use confidence-based selection such as MaxEns? Investigating Multi-Pivot Ensembling already shows it trails English pivoting on hallucinations. The confidence-ordered ablation checks whether that failure persists when the combiner is an LLM editor.
Proposed Experiments and Decisive Checks
We evaluate on the Flores-200 non-English directions used by SCALE, including Lao to Armenian, and on the 20 FLORES-101 low-resource directions of Investigating Multi-Pivot Ensembling. We report COMET, spBLEU, hallucination rate, prompt tokens and latency, and we check a sample of hallucination judgments by hand.
Three results would decide the question. First, the random-order loading curve must show a peak followed by rising hallucinations. The location of that peak must correlate across directions with mean reference-based error correlation, and $\rho$ must track that reference-based correlation. Second, in the injection study the editor's reproduction of a shared error must rise sharply with the fraction of drafts carrying it, while $\rho$-based admission keeps reproduction flat. Third, on held-out directions the $\rho$ rule must beat SCALE-pivot on COMET at equal or lower hallucination, reach within about one COMET point of the hindsight-oracle $K$, and beat minimum-Bayes-risk selection on the same pool.
If the curve is monotone, meaning more drafts always help, the agglomeration hypothesis is wrong for LLM editors, and that is itself worth reporting. If the curve peaks but $\rho$ fails to predict where, the lens holds but the measure needs replacing. Only the third outcome supports the method.
References
- Investigating Multi-Pivot Ensembling with Massively Multilingual Machine Translation Models
- SCALE: Synergized Collaboration of Asymmetric Language Translation Engines
- Simultaneous Multi-Pivot Neural Machine Translation
- Machine Translation by Triangulation: Making Effective Use of Multi-Parallel Corpora
- Pivot language approach for phrase-based statistical machine translation
- Analyzing the Use of Character-Level Translation with Sparse and Noisy Datasets
- Combined effect of α-nucleating agents and glass fiber reinforcement on a polypropylene composite: a balanced approach
- Effect of TiC Content on the Microstructure and Mechanical Properties of TiC/Fe Matrix Composites Fabricated by Spark Plasma Sintering
- Effect of Aggregation Structure on Fatigue Characteristics of (Nylon 6/Clay) Composites
- No Language Left Behind: Scaling Human-Centered Machine Translation