When a language model summarizes a news article in Turkish, Swahili or Amharic, it often has to reuse a name or number from the source in a different grammatical form. A city named in the locative case in the article may need the dative case in the summary. A copy mechanism lets the model point back at the source and duplicate what it finds there, and it is the standard way to stop a model from inventing entities. But current copy mechanisms duplicate exact tokens. In these languages the exact token is often the wrong one, and the tokenizer may also split the needed form into different pieces than the source form. We propose a decoding-time copy mechanism that works on character strings rather than token identities. It copies the stem of a source word, then hands control back to the model to write the grammatical ending. It requires no retraining of the base model.
The opportunity matters because two findings sit side by side in the literature without being connected. Large models hallucinate far more in lower-resourced, inflected languages: GPT-4o scores 5.51 on a meaning-entailment check in Farsi dialogue versus 93.56 in Mandarin (Investigating Hallucination in Conversations for Low Resource Languages). The obvious remedy, a pointer-generator, has a documented failure in exactly this regime, which its authors blame on subword tokenization (Pointer-Generator Networks for Low-Resource Machine Translation: Don't Copy That!). Our claim is that this failure comes from the unit being copied, not from copying itself. A pointer defined over character strings, with an explicit release point at the stem boundary, should recover entity fidelity that exact-token pointers lose.
Core Claim
A pointer-generator is a model that, at each step, mixes two choices: generating a word from its vocabulary, or copying a word from the input by following its attention weights. Existing pointers assume the copied unit has the same surface form in source and output. That holds in all of the following:
- LargePiG, which turns decoder-only LLMs into training-free pointer-generators and is evaluated only in English and Chinese.
- The classic copy layers surveyed in Pretrained Language Model for Text Generation: A Survey.
- A Copy-Augmented Generative Model for Open-Domain Question Answering.
At the other extreme, the morphology literature shows that copying stems is exactly what neural models fail to learn on their own. Transformer accuracy on unseen Finnish lemmas collapses to 5.7% (Can a Transformer Pass the Wug Test?). An explicit COPY action won the 100-example tier of SIGMORPHON 2017 at 50.6% (Align and Copy). Both results are for isolated single-word inflection, not running text.
We bring the stem-copy-then-inflect idea from single-word inflection into document-level LLM decoding. We claim it improves the correct-form rate of source entities specifically on cases where the required form differs from the source form. On cases where the form is unchanged, it should match exact-token pointers.
Mechanism
At decoding step $t$, let $g$ be the characters generated so far in the current word, and let $u_1,\dots,u_M$ be the source words. Each source word $u_j$ has an estimated stem boundary $b_j$, a character position. For a vocabulary token $v$ with string $\sigma(v)$, say $v$ is compatible with $u_j$ when $g\,\sigma(v)$ is a prefix of the first $b_j$ characters of $u_j$. Let $C_j$ be the set of such tokens. Let $\pi_t(j)$ be the self-attention mass from the last position to the tokens of $u_j$, normalized over source words. The output distribution is
$$P(v) = (1-\lambda_t)\,P_{\text{LM}}(v) + \lambda_t \sum_j \pi_t(j)\,\frac{\mathbf{1}[v \in C_j]\,P_{\text{LM}}(v)}{\sum_{v' \in C_j} P_{\text{LM}}(v')},$$
where $P_{\text{LM}}$ is the base model's next-token distribution and $\lambda_t\in[0,1]$ is a copy gate.
Each source word's pointer mass is spread over every token that continues its stem, weighted by the model's own preference. The pointer therefore does not care how the tokenizer chose to split the inflected output form. This targets the inconsistent-split failure diagnosed in the machine translation paper.
Copying releases on its own. Once the generated characters pass $b_j$ or diverge from every source stem, no token is compatible, the pointer term vanishes, and the model writes the ending.
We estimate the stem boundary without labeled morphology, since such resources are scarce for most target languages. We collect word-final character strings from unlabeled monolingual text that attach to many distinct prefixes. We then take $b_j$ as the start of the longest such suffix found on $u_j$.
For the gate $\lambda_t$ we compare two options. The first is LargePiG's training-free gate, a Jensen-Shannon divergence between high-layer and final-layer predictions. The second is a learned gate: a logistic function of a handful of decoder features, fit on a few hundred examples, in the spirit of the sub-1,600-parameter copy layer of the copy-augmented QA work.
The comparison tests an untested assumption. LargePiG's gate relies on factual tokens settling only at the final layer. That may not hold for under-trained low-resource tokens, whose embeddings are diluted in multilingual models (Building a Data Infrastructure for a Mid-Resource Language: The Case of Catalan; AfriqueLLM).
Research Program
Diagnostic. The work begins with a diagnostic, because the idea rests on an empirical premise: that entity errors in these languages are often inflection shifts rather than invented entities. We use XL-Sum, which reports weaker results on morphologically rich languages such as Amharic and Azerbaijani. We take those two plus agglutinative and inflectional languages from its set, such as Turkish, Swahili, Tamil, Hindi and Persian.
For each summary, we align every reference entity or number to its source occurrence by long shared prefix. We then split the alignments into an identical-form subset and an inflection-shift subset, and report how large the shift subset is per language. If shifts are rare, the mechanism has little room to help, and we will say so before building further.
Generation experiments. We then run open multilingual decoder models of the kind used in the hallucination study, Llama-3.1 8B and a Qwen model, prompted for summarization. Every method in a comparison uses the same model, prompt and beam settings.
Grounded question answering. As a second same-language grounded task, we use AfriQA. Answers there must be generated from retrieved passages. The paper reports hybrid retrieval recall@10 of 73.9%, so we condition on gold passages where available. This isolates generation from retrieval.
Translation contrast. Finally, we revisit the translation setting where pointers failed: Hindi-Marathi from the data used in the "Don't Copy That!" study. This tests whether string-level copying turns that negative result positive. It speaks directly to the open contradiction between that paper and summarization work finding pointers indispensable for entities (A Pilot Study of Domain Adaptation Effect for Neural Abstractive Summarization; Dr. Summarize).
Metrics. We report three groups:
- Entity fidelity, the primary metric: correct-form rate, stem-correct-but-wrong-ending rate, and invented-entity rate, each reported separately on the identical-form and inflection-shift subsets.
- Hallucination: an entailment score of the summary against the source, plus human annotation of XL-Sum's inconsistency and unsupported-information properties on a sample per language.
- Fluency: ROUGE, which serves as a check that fluency is preserved, not as evidence of faithfulness. The hallucination study shows ROUGE can stay respectable while entailment collapses.
Baselines
Unmodified decoding. The naive reference point is the base LLM with ordinary beam search and the same prompt. A fine-tuned mT5 summarizer, as in XL-Sum, gives a supervised reference. XL-Sum reports ROUGE-2 above 11 on ten languages, with some above 15. No per-language number for our chosen languages is given in the file, so we retrain it on identical splits. Every method must stay within one ROUGE-2 point of unmodified decoding while improving entity fidelity.
Exact-token pointer (LargePiG). This is the most direct competitor: the same idea with token identity as the copy unit. We run it on the same model with its own gate. No reported result found on Lacuna for LargePiG on any low-resource or morphologically rich language. The file reports only superiority on MC1–MC3 truthfulness in English and Chinese, at about 6% latency overhead on LLaMA-2-7B. Beating it means a significantly higher correct-form rate on the inflection-shift subset, with parity on the identical-form subset. Gains confined to the identical-form subset would mean we built a slower LargePiG.
Contrastive decoding. This is the strongest existing hallucination method in the file (Contrastive Decoding Reduces Hallucinations in Large Multilingual Machine Translation Models). It penalizes tokens favored by a source-weak copy of the model and scales the penalty down when the full model is confident. That paper reports a mean COMET gain of 14.6 ± 0.5 points on hallucinated sentences for M2M-100 418M and 11.0 ± 0.6 for 1.2B on FLORES-101. For decoder-only models we build the amateur by scaling down attention to the source span, with the same dynamic weighting. Its penalty strength gets the same tuning budget as our gate, on the same dev set. We also run contrastive decoding combined with the string pointer.
Ablations. Three ablations isolate the key ingredients:
- No release: set $b_j$ to the full word, so the pointer pushes toward copying the source ending. This should hurt on the inflection-shift subset.
- Gold boundaries: replace the unsupervised suffix inventory with gold segmentation where it exists, which bounds the cost of our cheap boundary estimate.
- Gate swap: swap the learned gate for the divergence gate, which tests whether LargePiG's layer-settling premise survives in low-resource languages.
Why not just…?
Why not just use contrastive decoding? This is the real threat, especially for the "generic response trap," where the model ignores the source and writes polite filler. Contrastive decoding attacks that directly, and a pointer gated off by a low $\lambda_t$ would not. Its limit is structural: it reranks the model's own distribution. If the correctly inflected entity is split into pieces the model rarely produces, the penalty cannot add probability there; it can only move it away from detached output. The experiments settle this. If contrastive decoding matches our correct-form rate on the inflection-shift subset, the pointer is unnecessary. We expect complementarity, with contrastive decoding winning on entailment and the pointer winning on entity form, and the combined run tests that.
Why not just fix the tokenizer and use a vanilla pointer? The "Don't Copy That!" authors suggest morphologically principled segmentation as the fix, and Unigram segmentation helped Azerbaijani translation (Benchmarking Azerbaijani Neural Machine Translation). But retokenizing means retraining or re-embedding a pretrained LLM, which is costly when data is scarce. Even a perfect morpheme tokenizer still needs a pointer that knows to stop before the ending. The gold-boundary ablation approximates this alternative inside a fixed model. If gold boundaries fail to beat unsupervised ones, tokenization is not the bottleneck.
Why not just fine-tune with substring augmentation? Synthetic stem data raised unseen-lemma accuracy in Finnish from 5.7% to 68.5% and in Ga from 49.2% to 100% (Can a Transformer Pass the Wug Test?). However, this was shown only for single-word inflection and requires fine-tuning. We run it as a LoRA-style fine-tune of the same model on the same data budget. A tie on entity fidelity would favor augmentation for its simplicity, and we would report that.
Why not just use a bigger model? The file's evidence argues against it: GPT-4o's 5.51 entailment score in Farsi, and below 3 spBLEU for large multilingual models on Bantu and Dravidian languages in FLORES-101. We still include one frontier-model run prompted to preserve source names, as an upper reference.
Decisive Checks
The idea stands or falls on the split between the two subsets. If the string pointer beats LargePiG and contrastive decoding on the inflection-shift subset, ties on the identical-form subset, and the no-release ablation erases the gain, then stem-level copying is the active ingredient. If the gains are spread evenly across both subsets, the benefit is just copying, and the mechanism adds nothing. On Hindi-Marathi, a positive result where the exact-token pointer was flat would locate the cause of the earlier negative result in the copy unit. A flat result would support that paper's stronger conclusion that copying is not the bottleneck in translation. That would leave our claim confined to same-language grounded generation.
References
- Investigating Hallucination in Conversations for Low Resource Languages
- Pointer-Generator Networks for Low-Resource Machine Translation: Don’t Copy That!
- LargePiG: Your Large Language Model is Secretly a Pointer Generator
- Pretrained Language Model for Text Generation: A Survey
- A Copy-Augmented Generative Model for Open-Domain Question Answering
- Can a Transformer Pass the Wug Test? Tuning Copying Bias in Neural Morphological Inflection Models
- Align and Copy: UZH at SIGMORPHON 2017 Shared Task for Morphological Reinflection
- Building a Data Infrastructure for a Mid-Resource Language: The Case of Catalan
- AfriqueLLM: How Data Mixing and Model Architecture Impact Continued Pre-training for African Languages
- XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages
- AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languages
- A Pilot Study of Domain Adaptation Effect for Neural Abstractive Summarization
- Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structures
- Contrastive Decoding Reduces Hallucinations in Large Multilingual Machine Translation Models
- Benchmarking Azerbaijani Neural Machine Translation
- The Flores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation