Pointing Into Retrieved Translations: Decode-Time Entity Grounding for LLM Translation

Large language models translate fluent sentences but often get names wrong. A film title gets translated word by word instead of being given its official local title, a person's name is "translated" by meaning, or a correct translation that sits right in the prompt is ignored. The usual fix retrieves the official target-language name from a multilingual knowledge graph such as Wikidata and pastes it into the prompt. This works well only when an upstream entity tagger has already found the right entity. On the SemEval-2025 entity-aware translation task, systems given gold entity identifiers reach 89.1 manual entity translation accuracy (M-ETA), while the best fully automatic pipeline reaches 77.1 (SemEval-2025 Task 2: Entity-Aware Machine Translation). Most of the remaining error comes from the brittle detection step.

We propose to move entity detection out of the preprocessing stage and into decoding. We retrieve candidate target-language names for every plausible span in the source, deliberately over-generating, so the candidate list contains many wrong entries. A training-free pointer then lets the model copy a candidate span token by token only when its own internal signals say it is about to produce a factual or entity token. A pointer-generator is a decoder that, at each step, mixes "generate a word from the vocabulary" with "copy a token from a given text." Our pointer copies from retrieved target-language names, not from the source sentence. Because those names are already in the target language and the target tokenizer, copying is well defined. Because the copy is soft, the model can still add case endings and other grammatical inflections. The two-sentence version is this: put many candidate translations of possible entities into the context, and let a pointer gated by the model's own uncertainty copy from them only when needed. Entity detection then becomes a soft, per-token decision that can tolerate wrong candidates, instead of a hard upstream filter whose misses starve the translator.

Core Claim

Soft pointing over a noisy, high-recall memory of retrieved target-language names should close a substantial part of the gap between automatic and gold entity detection, without the grammatical damage caused by placeholder substitution or forced string insertion. The claim is specific to the regime where candidates are noisy. With clean gold candidates, plain prompting is already strong, and we expect small gains there. With over-generated candidates, prompting and hard constraints should both degrade. Prompting can get distracted by spurious names, and hard constraints force wrong names into the output. A gated pointer should degrade gracefully.

Relation to Prior Work

The mechanism comes from LargePiG: Your Large Language Model is Secretly a Pointer Generator. LargePiG turns an off-the-shelf decoder-only model into a pointer-generator without training. It takes the copy distribution from self-attention, and it sets the copy probability from the Jensen-Shannon divergence between intermediate-layer and final-layer predictions. The motivation is that factual tokens shift late, when feed-forward memory engages, while function words settle early. LargePiG was tested only on monolingual query generation, copied only exact strings from the input document, and used models up to 7B parameters. We keep its gate but redirect its pointer to a retrieved memory of target-language aliases. This is exactly the transformed-entity case it leaves open.

Three results constrain the design. First, Pointer-Generator Networks for Low-Resource Machine Translation: Don't Copy That! found that copying source subwords in neural translation gives at best marginal BLEU gains. It also found that copying is slightly worse on subsets with high shared-subword density, because the model copies linguistically meaningless BPE fragments. We avoid both problems by never copying from the source and by copying contiguous spans of alias tokens rather than isolated subwords. Second, End-to-end entity-aware neural machine translation showed that the classic entity-masking pipeline (replace with <PER_0>, translate, re-insert) lowers BLEU by stripping context. Third, the FII the Best system (FII the Best at SemEval 2025 Task 2) found that placeholders break target-language grammar, and Extract and Attend found the same for hard replacement. These results rule out designs that remove the entity from the model's view, so our model always sees the real source entity.

Extract-and-Attend is also the closest existing design. It puts retrieved target-language candidates in a decoder prefix and relies on ordinary self-attention to use them. It has no explicit copy gate, it works with small encoder-decoder models, and it adds latency (389ms to 624ms). KG-MT (Towards Cross-Cultural Machine Translation with Retrieval-Augmented Generation from Multilingual Knowledge Graphs) likewise injects Wikidata aliases through prompt text or prefix embeddings. Our method differs from both in two ways. It has an explicit, uncertainty-triggered copy gate over the memory. It is also evaluated under deliberately noisy candidate sets rather than curated ones.

The motivating evidence for a decode-time rather than prompt-level fix is twofold. Even oracle-ID systems leave about 11% of entities wrong (SemEval-2025 Task 2). CHILL (CHILL at SemEval-2025 Task 2) needed iterative GPT-4o self-refinement to fix entities that were present in the context but ignored.

Entity masking enters the proposal as a measurement tool, not as the method. We borrow mention randomization, the technique of replacing each entity with a unique random string so that no memorized association can help. We use it to build test items where only the retrieved memory can supply the right answer. This isolates faithfulness to context from recall of parametric knowledge, which is exactly the failure that the hallucination literature describes as fabricated grounding (Factual Inaccuracy and Hallucination in Large Language Models).

Method

For a source sentence, we enumerate candidate spans: tagger outputs at a low confidence threshold, plus capitalized and quoted n-grams and spans that match Wikidata labels. We retrieve each span's target-language label and aliases, and we place them in the prompt as a candidate memory $M$ of tagged spans. Each candidate is paired with its source span and entity type, but no candidate is marked as correct.

At decoding step $t$, the output distribution is

$$P_t(w) = (1 - g_t)\, P^{\text{vocab}}_t(w) + g_t \sum_{j \in M,\ m_j = w} \alpha_{t,j},$$

The terms are defined as follows. $P^{\text{vocab}}_t$ is the model's ordinary next-token distribution. $\alpha_{t,j}$ is the attention mass from the current position to memory token $m_j$, renormalized over $M$ and averaged over a small set of heads chosen on a development set. $g_t \in [0,1]$ is the copy gate. Following LargePiG, $g_t$ is a monotone function of the Jensen-Shannon divergence between an intermediate layer's predicted distribution and the final layer's. We add one cross-lingual change: the gate is also multiplied by the concentration of $\alpha_{t,\cdot}$ on a single candidate span. The model therefore copies only when it is both uncertain in the way factual tokens are and focused on one candidate.

Once copying begins inside a span, a span-continuation bonus raises the weight on the next token of that same span. This prevents the fragment-copying failure from Don't Copy That! When the span ends, the generator is free to add inflectional material, such as an Arabic definite article or a Turkish case suffix. The method needs no training. The only tuned quantities are the gate's scale, the choice of heads, and the choice of comparison layer, all selected on the SemEval development split.

Research Program

The work proceeds in three stages. First, we build the candidate retriever and measure its recall and precision against gold entities on XC-Translate. The aim is a recall close to the gold condition at precision we deliberately keep low. Second, we implement the gated pointer on an open model where attention is accessible. Our main choice is Qwen2.5 Instruct at 7B and 32B, the family behind the strongest SemEval systems. We tune the gate on the development split only. Third, we run a noise sweep and the randomized-entity probe described below. These test the central claim: that soft pointing is robust to spurious candidates in a way that prompting and hard constraints are not.

Baselines

The naive reference point is the same backbone translating with no retrieval. Its expected failure mode is literal over-translation. Published references for this regime are GPT-4 at 25.3% M-ETA and vanilla NLLB-200 at about 17.9% on XC-Translate (Towards Cross-Cultural Machine Translation with Retrieval-Augmented Generation from Multilingual Knowledge Graphs).

The strongest competitor is retrieval-augmented prompting, which puts the identical candidate memory in the identical prompt and lets the model decode normally. This isolates exactly one difference: the gate. It is the strongest competitor because it is what the top SemEval systems do. The best end-to-end system reported in SemEval-2025 Task 2 (Salt-Full-Pipeline) reaches M-ETA 77.1 and COMET 91.8, and the best gold-ID system (Qwen2.5-72B with LoRA) reaches M-ETA 89.1 and COMET 94.7. YNU-HPCC reports an average M-ETA of 94.506 for a Qwen NER-plus-translator method (YNU-HPCC at SemEval-2025 Task 2). The file does not let us confirm that this figure is on the same official test protocol, so we will rerun that configuration ourselves rather than compare against the number. We run prompting at both 7B and 32B with the same retriever, prompt template, and development-set tuning budget for prompt wording as the pointer gets for its gate. Beating it means a statistically significant M-ETA gain in the automatic-detection condition with COMET no lower. In the clean gold condition, we only aim to match it.

We also run two structural competitors. The first is placeholder delexicalization in the style of FII Strategy 1, the established masking pipeline. The second is hard lexically constrained decoding, which forces the selected alias string into the output. The file contains no reported result on Lacuna for constrained decoding on XC-Translate, so we will implement it ourselves. Both receive the same candidate memory. For constrained decoding, we must choose which candidate to force, and we use the retriever's top candidate per detected span. We also report CHILL-style two-round self-refinement on the same backbone, with its cost in model calls. CHILL reports gains of 0.19 to 1.66 points in the harmonic mean over retrieval alone.

Three ablations isolate the key ingredients. The first replaces the divergence-based gate with a constant gate. The second points at source tokens instead of the memory, which is LargePiG used unchanged. The third removes the span-continuation bonus, which tests whether we have truly escaped the fragment-copying failure.

Why not just…?

Why not just prompt with the retrieved aliases, as the SemEval winners do? This is the real threat. With gold candidates it may simply be as good, because 89.1 M-ETA leaves limited headroom. The proposal's claim lives in the noisy-candidate regime. The noise sweep settles it: if prompting's M-ETA does not fall faster than the pointer's as spurious candidates increase, the pointer adds nothing, and we will report that.

Why not just force the alias with constrained decoding? Forcing guarantees the string appears. However, it needs a hard choice of which candidate to force, so it inherits the detection bottleneck. It also cannot inflect the name, which is the failure documented for hard replacement by Extract and Attend. We expect constrained decoding to be competitive for Chinese, Japanese, and Thai, where names rarely inflect, and to lose for Arabic, Turkish, and German. Our per-language breakdown is designed to show exactly this split. If constrained decoding wins on the inflecting languages too, our fluency argument fails.

Why not just iterate self-refinement until entities are right? CHILL's reported gains are modest (0.19 to 1.66 points) and require multiple full model calls per sentence. LargePiG-style gating adds about 6% latency on LLaMA2-7B. We will report accuracy at matched wall-clock cost.

Why not just fine-tune with an entity-aware loss, as Xie et al. do? That approach still relies on parametric memory to produce the entity. Parametric memory is precisely what fails on long-tail and non-US entities: Cultivar attributes over 80% of locale-specific failures to named entities (Cultivar). The randomized-entity probe tests this directly, because no amount of fine-tuning can recall a fabricated name.

Proposed Experiments

The main evaluation is XC-Translate on all ten English-to-X directions. We report M-ETA, COMET, and their harmonic mean under three conditions: gold candidates, automatic high-recall candidates, and a controlled noise sweep. In the sweep, we add 0, 2, 5, and 10 distractor aliases per sentence, drawn from same-type Wikidata entities. The decisive result is the slope of M-ETA against the number of distractors for prompting, constrained decoding, and the pointer.

The second experiment is the randomized-entity probe. We take XC-Translate sentences, replace the gold entity with a fabricated name, and insert a fabricated target alias into the memory alongside real distractors. The correct output can then come only from copying. Any other output is a hallucination, either a literal rendering or a memorized real name. This probe directly measures faithfulness to the retrieved memory.

For locale robustness, we run Cultivar, translating into English with aliases retrieved for the localized entities. We report ΔBLEU and ΔchrF between localized and original sentences. The question is whether the pointer shrinks the localization drop, such as the ΔBLEU of −4.29 reported for Chinese localized to Singapore.

Finally, we check the gate itself. We measure whether high gate values coincide with entity tokens, and whether gate activation predicts entity errors in the prompting baseline. If gate activation does not separate entity from non-entity positions in the target language, the LargePiG signal does not transfer across languages, and the method's premise fails.