Language models often state false facts in fluent sentences. A simple test is to hide the suspicious part of a sentence, ask the same model to fill it back in several times, and see whether it keeps producing the same answer. If the model knows the fact, it should refill it consistently. If it invented the fact, the refills should scatter. This test breaks down when the hidden part is a whole phrase. A true phrase such as "a prestigious research university in Boston" can be refilled in dozens of valid ways, so honest variation in wording looks the same as fabrication. We propose to hide only the phrase's semantic head, the single word or name that carries its factual commitment ("Harvard", "1987", "bone"), and keep its modifiers in place. The modifiers then narrow the set of valid answers, so that a disagreement among refills signals a gap in the model's knowledge rather than a choice of synonym.
The opportunity comes from two observations in the hallucination literature. In HalluEntity, which annotated 18,785 entity spans in model-written biographies, errors concentrate in proper nouns (30.9% wrong) and numbers (36.2% wrong). The same work found that likelihood- and entropy-based detectors lose precision because true, informative entities also have high entropy. Classical work on syntactic heads supplies the tool for separating the two: once you find the word that governs a phrase, you can strip away the descriptive material that causes surface variation. Our proposal turns that tool into a black-box detector that needs no training and no retrieval.
Core Claim
Hallucination detection by refilling a hidden slot works much better when the slot is the semantic head of an entity phrase, with its modifiers left visible. Two mechanisms should drive the gain. First, the visible modifiers act as constraints, so true heads are refilled consistently even when their token entropy is high. Second, "transparent" heads are rerouted to the noun they introduce. These are grammatical governors such as "member", "one", "fragment" or "subset", which carry little factual content of their own. In "a member of the Labour Party" the slot becomes "Labour Party", not "member". The detector is scored with a single quantity. For an entity whose semantic head is $h$, we draw $K$ refills $\hat h_1,\dots,\hat h_K$ from the model given the sentence with $h$ hidden, and compute
$$ s(h) = 1 - \frac{1}{K}\sum_{k=1}^{K} \mathbb{1}\left[\mathrm{norm}(\hat h_k) = \mathrm{norm}(h)\right], $$
where $\mathrm{norm}$ lowercases text, canonicalizes numbers and dates, and strips titles. Higher $s$ means more likely hallucinated. Because the slot is short and tightly constrained, exact match after normalization is meant to replace the semantic-equivalence judge that whole-span comparison would need.
Relation to Prior Work
Three papers motivate this design.
- HalluEntity documents the confusion between entropy and truthfulness and the "end-of-sentence trap", where hallucination rates rise from 9% at sentence starts to 36% at sentence ends. Those end positions are where English clauses place their object heads. HalluEntity diagnoses these patterns after the fact. We use them to decide what to probe.
- Do Language Models Know When They're Hallucinating References? shows that indirect, repeated questioning exposes fabricated citations better than asking the model directly whether something exists. We carry that indirect-consistency idea from citations to arbitrary entity heads in free text.
- FActScore shows that precision falls on rare entities and in later sentences. It also splits text into atomic facts with an unconstrained prompt that ignores phrase structure. We keep its benchmark as a downstream measure but replace its unit of checking.
Four lines of syntactic work supply the method.
- Ponzetto et al. found that heads of category labels match taxonomies more cleanly than full strings.
- FALCON extracted headwords from short questions ("lighthouse" in "the lighthouse in Colombo") to link them to knowledge bases.
- GLARF and Romano et al. characterized transparent heads, where the grammatical governor ("box", "fragment") masks the true argument ("cookies", "protein X").
None of this work touches language models or hallucination. We reuse its head rules as a preprocessing step.
The closest systems that do connect these areas each differ in a specific way.
- DEPTH uses dependency paths to cut hallucinated relations on SciERC from 96.9% to 7.1%. It relies on reward-model training and PPO, a reinforcement-learning fine-tuning method, and it works only on classifying relations between given entity pairs. We use no training and work on open-ended generation.
- ZEROFEC generates questions from extracted information units, without head structure, and answers them against supplied evidence. We need no evidence and probe the model against itself.
- ToPro builds one fill-in-the-blank prompt per token. Our probing is selective: one slot per entity, which typically covers a small fraction of tokens.
- Generated Knowledge Prompting found that 86% of its errors trace to hallucinated self-generated statements. This is a natural downstream use for our detector, but we treat it only as a secondary test.
Research Program
The pipeline has three parts.
Head extraction. For each entity span in a generation, we run a dependency parser and take the governing word as the syntactic head. Proper names and numbers are treated as atomic heads, so "University of Michigan" is one slot. A transparent-head lexicon then reroutes heads such as "member", "one", "part", "group", "series", "fragment", "subset" and "number" to their prepositional object. The lexicon is seeded from the GLARF and Romano categories and extended from parser output on held-out biographies. Because parsers degrade on unusual text, we also build a zero-shot variant in which the model itself names the head of each span. This lets us measure what the parser actually contributes.
Refill prompting. We use a fixed zero-shot instruction. It shows the sentence with the head replaced by a blank and asks for the single most likely filler. To prevent leakage from the model's own writing, the surrounding context is the preceding sentences of the generation, never later ones. We draw $K$ samples at temperature 1.0, with $K$ fixed at 10 for the main runs and varied from 1 to 20 in a cost study.
Use of the score. The score serves two roles. As a detector, it is evaluated directly against entity labels. As a mitigation step, heads scoring above a threshold are either removed by a zero-shot rewrite that drops the claim or replaced by an explicit hedge, and the result is re-scored with FActScore.
We will also analyze where gains come from, broken down by entity type (proper noun, number, common noun), sentence position, and the subset of spans containing transparent heads.
Baselines
Naive reference points. Token likelihood and token entropy averaged over the entity span, as benchmarked in HalluEntity. The file gives no separate figures for them, so we will rerun them on the same model. A random scorer sets the AUPRC floor at the base hallucination rate, roughly 15%.
Strongest existing method. Focus, the uncertainty method in HalluEntity that selects keywords and propagates uncertainty through them. It is the strongest because it already tries to concentrate the uncertainty signal on informative tokens, which is our hypothesis in different form. HalluEntity reports AUROC 0.78 and AUPRC 0.40 for Focus on Llama3-8B. We will rerun Focus on Llama3-8B over the same entity annotations to confirm we reproduce this number before comparing. Beating it means a higher AUPRC at matched AUROC or better on the same model. The target is a gain of at least 0.05 AUPRC, consistent across three seeds of refill sampling. AUPRC is the right emphasis because the documented weakness is precision.
Direct query. A zero-shot prompt asking the model whether the sentence's claim about the entity is true, sampled $K$ times. This mirrors the direct-query condition of Agrawal et al., which their indirect probes beat. It checks that our gain comes from the slot design and not just from repeated sampling.
Ablations that isolate the key ingredient. All of these use the same model, the same $K$, and the same prompt template, and differ only in what is hidden.
- Whole-span refill: mask the entire entity span and judge equivalence with an LLM.
- Rightmost-token refill: the positional head heuristic of Chodorow et al..
- Model-chosen key token.
- Syntactic head without transparent rerouting.
Prompt wording for every method gets the same small tuning budget of five templates, selected on a held-out split of biographies. We will report token cost per biography for every method.
Why not just…?
Why not just mask the whole entity span and let an LLM judge whether the refills mean the same thing? This is the most serious threat, and we do not dismiss it. Its weak point is the confusion HalluEntity documents: an unconstrained span has many valid fillers, so a judge must decide equivalence among loosely related answers. That adds a second model, and possibly a second source of error, to every comparison. The whole-span ablation is run at matched sample count and judge budget. If it matches head-slot refill on AUPRC, we will conclude that syntax adds nothing beyond masking. The contribution then shrinks to a cost result, since exact-match scoring is cheaper than judge calls, and we will report it that way.
Why not just take the rightmost noun, or let the model pick the key token, instead of parsing? For flat proper names and numbers, the rightmost-token rule may well suffice. The rightmost-token ablation will show it. The parser is expected to matter on possessives, prepositional attachments and transparent heads. Lee et al. needed rules that navigate syntax precisely because adjacent modifier nouns get picked by mistake. If neither the parser nor the rerouting step beats the heuristics on the transparent-head subset, we drop them.
Why not just use token uncertainty such as Focus? HalluEntity reports that likelihood-style methods confuse lexical diversity with fabrication and stop at AUPRC 0.40 even with Focus. They also require access to the model's internal probabilities, whereas refill probing works on any API. The direct comparison on the same model settles which is more precise.
Why not just retrieve evidence and verify against it? Retrieval is a different regime and remains a fallback, not a replacement. FEVER reports end-to-end accuracy of 31.87% with retrieved evidence, against 64% to 88% with gold evidence, and evidence recall of 45.89%. FActScore reports that retrieval-augmented PerplexityAI still reaches only 71.5%. We include FActScore's own retrieval-based checker as an upper reference in the mitigation experiment, not as a competitor.
Proposed Experiments
Main detection experiment. This runs on HalluEntity's 157 annotated biographies. Llama3-8B is the shared model with Focus, and we add a second, black-box chat model, since HalluEntity found that model scale has little effect on detection. Metrics are AUROC and AUPRC at the entity level. Results are broken down by proper noun versus number versus common noun, and by sentence-start versus sentence-end positions. The test is decisive on two counts. Head-slot refill must beat Focus on AUPRC overall. It must also cut false positives on true entities whose token entropy is in the top quartile; this is exactly the population where entropy methods fail, and it is where our mechanism predicts the gain.
Transparent-head experiment. We will hand-annotate the biography spans containing transparent heads, expected forms being "one of the founders of", "a member of" and "a series of". On this subset we compare rerouted and unrerouted slots. As a stress test outside biographies, we also run on biomedical generations built from the Romano et al. setting, where "fragment of X" constructions are common. The file reports no prior hallucination-detection numbers on this subset, so this experiment is exploratory.
Mitigation experiment. We generate biographies for the FActScore entity list, apply the detector with abstention, and report FActScore together with the number of supported facts retained, so that gains cannot come from simply deleting content. The reference point is ChatGPT's reported FActScore of 58.3%. The decisive outcome is a precision gain that is larger on rare entities than on frequent ones, at no greater loss of supported facts than whole-span or Focus-based filtering at the same threshold.
Cost study. We plot AUPRC against total probe tokens for $K$ from 1 to 20, for all methods.
References
- HalluEntity: Benchmarking and Understanding Entity-Level Hallucination Detection
- Do Language Models Know When They're Hallucinating References?
- FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
- Knowledge-Rich Word Sense Disambiguation Rivaling Supervised Systems
- FALCON: An Entity and Relation Linking Framework over DBpedia
- Automatic Recognition of Logical Relations for English, Chinese and Japanese in the GLARF Framework
- Investigating a Generic Paraphrase-Based Approach for Relation Extraction
- Hallucination-Resistant Relation Extraction via Dependency-Aware Sentence Simplification and Two-tiered Hierarchical Refinement
- Zero-shot Faithful Factual Error Correction
- ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks
- Generated Knowledge Prompting for Commonsense Reasoning
- Extracting Semantic Hierarchies from a Large On-Line Dictionary
- High Accuracy Rule-based Question Classification using Question Syntax and Semantics
- FEVER: a Large-scale Dataset for Fact Extraction and VERification