Systems that point to the exact words in an LLM answer that are unsupported or false have plateaued at modest scores. The best reported detector reaches 55.9 Span-F1 on RAGTruth. Our hypothesis is that a substantial part of this remaining error comes from boundary placement rather than from detection itself. A detector often finds the right claim but draws its edges wrong. It may include a determiner, drop a unit, or stop one word short of a multi-word name. Current training scores such near-misses exactly like a completely missed hallucination.
Named entity recognition, the task of finding and typing names in text, faced this problem years ago and addressed it. The fix was to train span models on soft targets that give some credit to spans near the gold one, and to add small classifiers that nudge each span edge left or right. We propose to import those two tools into hallucination span detection. We will first measure how much of the current error is really boundary error, and then test whether boundary-aware training recovers it cheaply.
Core Claim
The claim is that hallucination span detectors trained on hard, exact span targets waste capacity on a boundary convention that the annotations themselves only loosely fix. Two changes should raise span-level scores without changing how well the detector decides whether an answer contains a hallucination at all. The first is to replace the hard targets with spatially smoothed ones. The second is to add a directional boundary-refinement step on top of any detector.
The new part is the error decomposition plus the boundary-aware training objective for this task. Neither has been applied to hallucination spans in the work we found.
Where This Sits
Three lines of work motivate the proposal by showing the span-level ceiling. RAGTruth annotated hallucinated spans in 17,790 RAG responses. It showed that a fine-tuned Llama-2-13B critic reaches 52.7% span-level F1 against GPT-4's 28.3%, while scoring 78.7% at the response level. That large gap between response-level and span-level scores is the first hint that localization, not detection, is the weak link. Learning to Reason for Hallucination Span Detection trains Qwen2.5-7B with group-relative reinforcement learning to reason before emitting a list of spans, reaching 55.9 Span-F1. The authors note formatting and boundary slips in the reasoning output. The Hallucination Detectives system on Mu-SHROOM takes span edges from a semantic role parser and reaches only 0.358 intersection-over-union in English, with lower scores in Arabic. This is direct evidence that fixed external boundaries align poorly with where annotators mark hallucinations. None of these three changes the training target to reflect boundary ambiguity. They treat spans as fixed classification targets, which is precisely the assumption we relax.
Two papers supply the method. Boundary Smoothing for Named Entity Recognition scores every candidate (start, end) pair in a grid. It moves a small fraction of probability mass from the gold span to its grid neighbours, weighted by Manhattan distance. This improves both F1 (93.65 on CoNLL 2003) and calibration. The paper also cautions that it can slightly lower recall, a constraint we must monitor. Negation Scope Refinement via Boundary Shift Loss adds heads that predict whether each token lies left or right of the nearest true start and end. This raises exact-match F1 on Sherlock from 84.19 to 87.10. We reuse both techniques unchanged in spirit and apply them to a different kind of span.
Two further papers show the problem is general. BenchIE found that lenient overlap metrics overstate open information extraction F1 by about 26 points because boundary errors are hidden. A Closer Look at Claim Decomposition shows that factuality scores move with how claims are cut up. Both confirm that boundary choice silently changes factuality measurement, which is why our decomposition analysis is worth doing on its own.
Research Program
The first step is diagnostic. We will take the predictions of several detectors on the RAGTruth and Mu-SHROOM test sets: the RAGTruth-style fine-tuned critic, an SFT Qwen2.5-7B, and the RL4HS recipe, retrained by us. We will classify every error into one of four kinds. A missed hallucination has no overlapping prediction. A false alarm is a prediction with no overlapping gold span. A boundary error overlaps a gold span but does not match it. A split/merge error occurs where one gold span is covered by several predictions, or the reverse. Boundary errors will be further split into which side is off and by how many tokens.
We will then recompute span-F1 under three credit rules: exact match, token overlap, and a relaxed rule allowing a one-token tolerance at each edge. Where RAGTruth provides annotations from more than one annotator, we will estimate how much annotators themselves disagree on edges. That disagreement estimate sets a ceiling on what boundary fixes can buy.
The second step is the method. We will build a span-grid detector on the same Qwen2.5-7B backbone used by RL4HS. The model reads the source documents and the response, and a biaffine scorer (a bilinear layer that scores each (start, end) token pair) is placed over the response's final-layer states. It is trained with boundary-smoothed targets as in the NER work. Following the smoothing formulation, each gold hallucinated span keeps mass $1-\epsilon$. The remaining $\epsilon$ is shared among grid cells within Manhattan distance $D$ of it, where $\epsilon$ is the smoothing mass and $D$ is the neighbourhood radius. Both are tuned on the development split.
We will add the left/right start and end shift heads from the negation-scope work. At inference they act as a refinement pass that can move each predicted edge. The shift heads can also be attached as a standalone post-hoc refiner to any detector's output spans, including RL4HS's list of spans. This tests whether boundary repair helps detectors we did not train.
A smaller third study tests the claim that boundary ambiguity also contaminates the uncertainty signals used by generation-time methods. HalluEntity scores fixed, pre-segmented entity spans with measures such as Focus (AUROC 0.78 on Llama3-8B). DRAD cuts generation at the start of an uncertain entity before retrieving. Both assume the entity edges are given. We will shift HalluEntity's entity spans by one or two tokens in each direction and measure how much Focus and entropy AUROC change. If scores move substantially, boundary choice is a hidden variable in entity-level uncertainty detection as well. That would motivate future work on rollback points for methods like DRAD, which we do not attempt here.
Baselines
The naive reference point is a detector that predicts no spans, together with one that flags every sentence containing a named entity or number. The first shows how imbalanced span-F1 is. The second shows how much of the score comes from the fact, reported by HalluEntity, that proper nouns and numbers carry most hallucinations.
The strongest existing method is RL4HS-7B from Learning to Reason for Hallucination Span Detection, with a reported RAGTruth Span-F1 of 55.9. It is the strongest competitor because it uses the same 7B backbone class and already optimizes a span-level reward. Any gain we show must survive comparison against a model that is explicitly rewarded for spans. We will retrain it with the published recipe on the same training split, and on our hardware we will report our reproduced number next to the reported 55.9. Beating it means exceeding our reproduced RL4HS Span-F1 under exact matching, with confidence intervals from bootstrap resampling over responses. Response-level F1 must not drop.
The supervised fine-tuned baseline is Qwen2.5-7B fine-tuned to emit spans, which the RL4HS paper reports at 50.1 Span-F1. The fine-tuned Llama-2-13B critic from RAGTruth, at 52.7% span-level F1, serves as a larger-model reference. For Mu-SHROOM English, the reference is the parser-based pipeline of Hallucination Detectives at 0.358 IoU.
The key ablation is the same span-grid model with identical backbone, data, and step budget, trained on hard targets ($\epsilon=0$) with no shift heads. This isolates the contribution of smoothing, and a separate run adds the shift heads only. All trained systems get the same number of hyperparameter trials on the development split, and all use the same number of GPU-hours, within a factor we will report.
Why not just…?
Why not just report a lenient overlap metric and declare the problem solved? Lenient scoring changes the number, not the detector. BenchIE shows that overlap metrics hide real errors: a span that swallows half a sentence around a wrong date gets credit it should not. Our decomposition reports exact, overlap, and tolerance-based scores side by side. The method must improve the exact score while reducing the gap between exact and lenient scores. If it only improves the lenient score, we have learned nothing.
Why not just snap predicted edges to word or phrase boundaries with a heuristic or a parser? This is a real threat for the easy cases of stray punctuation and determiners, so we include it as a baseline. We will apply a rule that trims function words and punctuation and extends to the end of the containing noun phrase to every detector's output. The parser-based Mu-SHROOM system, however, shows the limit of fixed syntactic edges: its 0.358 English IoU suggests annotated hallucination spans often do not coincide with parser constituents. If snapping matches the learned shift heads, we will say so and conclude that the boundary errors are mostly superficial.
Why not just give RL4HS a softer overlap-based reward instead of changing the architecture? This is the most serious alternative, because reward shaping could achieve the same effect as smoothed targets. We will run it directly, replacing RL4HS's span reward with an IoU-based reward at matched compute. If it equals our smoothed span-grid model, the contribution reduces to the diagnosis plus a cheaper training recipe. That outcome would still be useful, since supervised smoothing avoids RL cost, and we will report it as such.
Why not just prompt a frontier model? The file's evidence already answers this for detection. GPT-4 reaches 28.3% span-level F1 on RAGTruth and GPT-o3 reaches 51.2 Span-F1, both below trained 7B models. We include GPT-4o-mini-style prompting only as a reference and do not expect it to be competitive on boundaries.
Proposed Experiments
The main experiment trains and evaluates all systems on RAGTruth's three tasks (question answering, data-to-text, summarization) and on Mu-SHROOM English and Arabic. We will report exact Span-F1, token-overlap F1, IoU where the benchmark defines it, and response-level F1. We will also report the four-way error breakdown for each system, which shows where each system's errors fall.
The decisive checks are three.
- Is the gap actually boundary error? If boundary-type errors make up only a small share of RL4HS's errors, the premise fails and the proposal becomes a negative diagnostic result.
- Does smoothing improve localization without changing detection? Smoothing should raise exact Span-F1 while leaving response-level F1 unchanged. If both rise, smoothing is acting as a general regularizer, not a boundary fix. We will test this by comparing against an ordinary label-smoothing control that spreads mass uniformly rather than to spatial neighbours.
- Does the refiner transfer? The post-hoc shift-head refiner applied to RL4HS outputs should close part of their boundary-error share. Gains should be largest on the summarization and data-to-text tasks, where numbers and multi-word names dominate.
Because smoothing is known to trade some recall for precision, we will report precision and recall separately at every step.
The HalluEntity boundary-perturbation probe is reported alongside. A drop in Focus AUROC under one-token span shifts would show that entity-level uncertainty detectors inherit the same boundary sensitivity. A flat result would limit our claims to supervised span detectors.
References
- RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models
- Learning to Reason for Hallucination Span Detection
- Hallucination Detectives at SemEval-2025 Task 3: Span-Level Hallucination Detection for LLM-Generated Answers
- Boundary Smoothing for Named Entity Recognition
- Negation Scope Refinement via Boundary Shift Loss
- BenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation
- A Closer Look at Claim Decomposition
- HalluEntity: Benchmarking and Understanding Entity-Level Hallucination Detection
- Mitigating Entity-Level Hallucination in Large Language Models