Language models are now routinely used as judges that grade other models' outputs. They are good at saying whether an answer contains a hallucination and poor at saying where it is. On the RAGTruth corpus, prompted GPT-4 reaches 63.4% F1 at deciding whether a response contains a hallucination but only 28.3% F1 at locating the hallucinated spans (RAGTruth). The best current fix asks a reasoning model to rewrite the whole text with inline error tags. That works for o3 and slightly hurts GPT-4.1 (Tagged Span Annotation). This proposal separates the two jobs the judge is currently forced to do at once. The judge first reasons in free text about what is wrong. It then emits one tag per word of the evaluated text in the BIO scheme, a labeling convention where each word is marked Beginning of an error span, Inside one, or Outside. Decoding is constrained so that the words themselves are copied by force and only the tag is free.
Because the judge chooses among just three tags at each word, its probabilities over those tags form a per-word distribution. That distribution can be compared directly against the soft, disagreement-weighted human labels used by the Mu-SHROOM hallucination benchmark. This gives span localization, boundary uncertainty, and a controlled test of why tagging helps reasoning models and hurts plain instruction models, all from open-weight models.
Core Claim
We claim that the failure of standard LLM judges at span localization comes mostly from making one decoding pass do three things at once: decide what is wrong, reproduce the text faithfully, and place boundaries. The deciding is the part LLMs are good at.
The proposed method removes the other two burdens. Forced copying removes text reproduction entirely, so rewrite drift cannot happen. Restricting the output at each word to three tags turns boundary placement into a small classification. Under this interface, three things should follow. First, non-reasoning open models given a short reasoning step should recover most of the gain Tagged Span Annotation gives reasoning models. Second, the gap in the Tagged Span Annotation results should largely close. Third, the tag probabilities should track human boundary disagreement better than hard spans or sampled-and-averaged spans.
The closest existing work frames inline tags as an "externalized intermediate reasoning step" (Tagged Span Annotation). Our hypothesis is the opposite reading. Reasoning models succeed despite the rewrite burden, because they deliberate beforehand. Non-reasoning models fail because of that burden. The experiments are designed to tell these two readings apart.
Research Program
The output interface works as follows. The judge receives the source or evidence, the evaluated text, and a rubric. It writes a free-text reasoning block of bounded length. It then enters a tagging phase. For each word of the evaluated text (whitespace or language-appropriate word segmentation, so tags never split subwords), the decoder is allowed only the tokens B, I, O (optionally typed, e.g. B-conflict). The word's own tokens are then appended by force. Illegal transitions such as O followed by I can be masked out, which is the standard BIO validity rule.
We study two readout modes:
- Sequential readout decodes tags left to right, with each tag conditioned on the previous ones. We record the conditional probabilities along the greedy path.
- One-pass readout places a tag slot after every word in a single prompt and reads the tag distribution at every slot in one forward pass. This is cheaper, but no tag sees earlier tags.
The per-word soft score is $p_i = P(B) + P(I)$ at word $i$. This is exact for one-pass readout and an approximation for sequential readout, since exact marginals would require summing over tag histories. We will report that approximation's effect by comparing against Monte Carlo estimates from 20 sampled tag sequences on a subset.
The program has two stages. The first is zero-shot: open instruction models (Qwen2.5 and LLaMA-3.1, the families used in Distributional LLM-as-a-Judge) plus one open reasoning-trained model, each with and without the reasoning block. The second is light parameter-efficient fine-tuning on PsiloQA's 63,792 multilingual span-tagged samples (PsiloQA). Its loss is cross-entropy on tag positions only. On data with soft labels, we add a per-word KL term toward the human distribution. This extends the distributional objective of Distributional LLM-as-a-Judge, which works only on sentence-level labels, down to boundaries.
Each prior line of work plays a distinct role here:
- Motivation. RAGTruth, Mu-SHROOM (annotators disagree substantially on span onsets and offsets), and the gap between average precision and IoU in PsiloQA all show that boundaries, not detection, are the unsolved part.
- Constraints on design. Direct character-index prompting loses nearly 5 F1 to inline tagging, which rules out offset outputs. The generative judge's lag behind BIO encoders in code-mixed NER, where Gemini Pro scores 62.24 F1 against HingBERT's 79.74 (Code-Mixed NER study), points to unconstrained generation as the problem.
- Borrowed method. Classic BIO sequence labeling with validity constraints comes from work such as Parsing Argumentation Structures.
Baselines
The naive reference point is a prompted judge without any structure. The same open model, given the same rubric, outputs a JSON list of erroneous substrings, which we map back to the text by string matching. On RAGTruth, the published analogue is prompted GPT-4 at 28.3% span F1 (RAGTruth). Our method should clear this by a wide margin even zero-shot. If it does not, the interface is not helping.
The strongest prompting competitor is Tagged Span Annotation: rewrite the text with numbered inline tags plus JSON metadata (Tagged Span Annotation). We run it on the identical open models with the identical rubric and reasoning budget, and we also cite its reported numbers. On WMT24 error-span annotation, o3 with Tagged Span Annotation reaches character-level F1 27.05, against XCOMET-XXL at 17.30 and GEMBA at 16.50. The file gives no numeric GPT-4.1 score, only that it was "slightly worse" than simpler formats. Beating this baseline means higher span F1 or IoU on the same open model at equal or lower output-token cost.
The strongest trained competitors fall into two families:
- Fine-tuned encoder. An mmBERT token classifier trained on PsiloQA, which its authors report outperforms uncertainty methods and LLM prompting (PsiloQA). No per-benchmark numeric result is found on Lacuna for this model, so we retrain it on the identical PsiloQA split with a matched tuning budget of 20 hyperparameter trials for every trained system.
- Fine-tuned decoder detector. The fine-tuned Llama-2-13B detector on RAGTruth, reported at 52.7% span F1. We compare against this number on the RAGTruth test split and fine-tune our judge on the RAGTruth training split for that comparison.
On Mu-SHROOM, the reported pipeline reference is the Hallucination Detectives system at English IoU 0.358 and Spearman correlation 0.322, and Arabic IoU 0.28 (Hallucination Detectives).
Three ablations isolate the key ingredient:
- The same fine-tuned model with the reasoning block removed. This amounts to a decoder with a word-level tag head, which isolates reasoning.
- The reasoning block kept but the tags produced by free rewriting rather than forced copying. This isolates the copy constraint.
- BIO transition masking removed. This isolates the validity grammar.
Why not just…?
Why not just fine-tune an encoder with a BIO head? This is the most serious threat. PsiloQA shows a fine-tuned mmBERT beating LLM prompting, and encoders are cheap and always produce valid tags. The evidence that encoders plateau is indirect. XCOMET-XXL trails o3 with Tagged Span Annotation on translation errors (17.30 vs 27.05), and encoders cannot read a long evidence document and reason over it. We settle the question head-to-head with the retrained mmBERT on identical data. We report results separately on the Mu-SHROOM surprise test languages and on RAGTruth responses whose supporting evidence spans more than one passage, where reasoning should matter most. If the encoder matches us everywhere, the honest conclusion is that span localization does not need an LLM judge, and we will report it that way.
Why not just use Tagged Span Annotation with a reasoning model? It already works for o3. But it depends on closed models, costs about $139 per 40k segments, and fails on non-reasoning models. Our reasoning-plus-rewrite ablation and the Tagged Span Annotation baseline on the same open model directly test whether forced copying is what rescues non-reasoning models.
Why not sample the span output several times and average for soft labels? Self-consistency voting is a genuine competitor for the Spearman metric. We run Tagged Span Annotation and JSON prompting with 10 samples each, at matched total token budget, and compare their per-word vote rates against our single-pass tag probabilities.
Why not classify whole sentences, or flag low-probability tokens? Sentence labels discard the within-sentence localization that makes spans useful (Hallucination Detectives). Raw token confidence was insufficient on Mu-SHROOM and lost to supervised span models in PsiloQA. We include both as cheap references, not as main competitors.
Proposed Experiments
We use four benchmarks:
- Mu-SHROOM: IoU plus Spearman correlation against soft human labels across its 14 languages (Mu-SHROOM).
- RAGTruth: span F1 and response-level F1.
- PsiloQA test split: 2,897 samples, IoU and average precision.
- WMT24 error-span annotation: character-level F1, as a transfer test outside hallucination detection.
We also report output tokens per example as a cost metric.
The first decisive check targets the tagging paradox. In the zero-shot setting on WMT24 and RAGTruth, forced-copy tagging with a reasoning block should lift the non-reasoning models above both their JSON and their Tagged Span Annotation scores. It should also narrow the gap to the reasoning model by at least half. If the gap persists, the "externalized reasoning" reading is supported and ours is wrong.
The second check targets boundary uncertainty. On Mu-SHROOM, per-word tag probabilities should achieve higher Spearman correlation than sampled-vote baselines at matched cost. The KL-fine-tuned judge should improve correlation without lowering IoU.
The third check is against the encoder. After fine-tuning on PsiloQA, the judge must beat the retrained mmBERT on IoU in at least the surprise languages and on multi-passage RAGTruth cases. On RAGTruth, it should approach or exceed the reported 52.7% span F1.
The fourth check tests the readout approximation. Sequential and one-pass readouts should rank within noise of each other, and both should agree with Monte Carlo marginals on the subset. If so, the cheaper one-pass mode is sufficient.
References
- RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models
- Tagged Span Annotation for Detecting Translation Errors in Reasoning LLMs
- Distributional LLM-as-a-Judge
- When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQA
- SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes
- Comparative Study of Pre-Trained BERT and Large Language Models for Code-Mixed Named Entity Recognition
- Parsing Argumentation Structures in Persuasive Essays
- Hallucination Detectives at SemEval-2025 Task 3: Span-Level Hallucination Detection for LLM-Generated Answers