When a language model states something false, the false material does not float free in the text. It is attached to the sentence in one of two ways. It may fill a slot the main verb requires, such as the person who won or the thing that was founded. Linguists call this an argument. Or it may hang off the sentence as optional decoration, such as "in 1987," "a leading," or "who later moved to Paris." Linguists call this a modifier. Formal grammars have long treated these two kinds of attachment as different operations: arguments are plugged into slots, while modifiers are attached beside existing material without disturbing it. This proposal asks whether hallucinations follow the same split, and whether editing them should too. The working conjecture is that much of the "invented" and "unverifiable" content that current editors handle worst is modifier material. Such content can be repaired by cutting it out. False arguments, by contrast, cannot be deleted without breaking the sentence. They have to be replaced from evidence, or the editor has to abstain.
Concretely, the plan is to relabel existing fine-grained hallucination benchmarks with one extra attribute per error span: whether it attaches as an argument or as a modifier. The first goal is to measure whether this attribute predicts how an error should be fixed and how much it matters. The second is to build an editor that may only perform the matching inverse operations: remove a modifier subtree, refill an argument slot from retrieved evidence, or flag and abstain. If the conjecture holds, this restricted editor should match free rewriting on factual precision while destroying much less correct content around the error. That second property is what current string-rewriting editors do not measure and do not protect.
Core Claim
Current taxonomies sort errors by what they are about. FAVA's categories are entity, relation, contradictory, invented, subjective and unverifiable (Fine-grained Hallucination Detection and Editing for Language Models). ZINA's are object, attribute, number, text, relation and fact (ZINA). Other taxonomies sort errors by pipeline stage, as in MedCalc-Bench. None records how the false span is attached to its host clause.
We claim that attachment type is a second axis, orthogonal to the content categories, and that it determines the correct repair. A false modifier can be removed and the remaining sentence stays grammatical and true. A false argument leaves a gap that must be filled or the whole claim withdrawn. The claim is falsifiable in a specific way. If attachment type carries no information about which repair human editors choose, or about how much correct content free rewriting destroys, the axis is useless.
The grammatical distinction comes from the formal-grammar literature, which here supplies the method rather than motivation. D-Tree Grammars separate attaching arguments from attaching modifiers. The latter is "sister-adjunction," which hangs a modifier tree beside an existing node instead of splitting the structure. Arguments and Modifiers from the Learner's Perspective uses the same split inside a probabilistic tree grammar. We reuse the operations and their inverses, not the parsers.
What the closest work does and does not do
FAVA's editor is the motivating case. It reports that unverifiable statements are over 40% of human-annotated hallucinations and are hard to fix because retrieval returns no evidence. Our proposal reads this differently: if most such content is modifier material, no evidence is needed to remove it. FAVA's synthetic training data is produced by prompting a model to insert errors into text. We keep that idea but make the insertion operation-typed, so each synthetic error is either a substituted argument or an adjoined modifier.
ZINA builds a graph over spans to keep cascading errors consistent and separates detection from rewriting to avoid the model drifting while it copies text. Its edits are still flat string replacements. Our dependency structure comes from the parse rather than an engineered graph. Our edit space is also a small set of tree operations rather than free text.
A Closer Look at Claim Decomposition shows that atomic-fact metrics such as FActScore move with the decomposition step. It finds that a decomposition based on event semantics, which splits sentences into events and their participants, scores best on DecompScore (42.3 versus about 32 for FActScore's own decomposer). This paper constrains our evaluation. Every precision comparison must hold the decomposer fixed, and it supports using a predicate–argument decomposition, which aligns naturally with our argument/modifier split.
Bergen et al. also constrain the design. Their unsupervised identification of modifiers reaches 62% precision but only 0.15 recall. We therefore do not induce the distinction from raw text. We take it from supervised semantic role labelling, the task of tagging which phrases are the core participants of a verb, using PropBank-style core roles versus "modifier" roles. Off-the-shelf constituency and dependency parsers supply the subtrees.
Machine translation testing via syntactic tree pruning deletes subtrees to probe translation robustness. It uses deletion as a test, not as a typed repair of detected errors.
Research Program
The first stage is annotation and measurement. We parse every response in FAVABENCH, about 1,000 human-annotated spans. Each annotated hallucination span is then aligned to the smallest subtree that covers it. We label that subtree as an argument (a core role of its governing predicate), a modifier (a modifier role, adjectival or relative-clause attachment, or an adverbial), or a whole clause. We hand-verify a stratified sample of several hundred alignments to estimate how much error the parser introduces.
The analysis then asks three questions. How do attachment types distribute across FAVA's content categories, especially invented and unverifiable? When FAVA's gold edits are made, how often does deleting the subtree recover the human-corrected sentence for modifiers versus arguments? Is a reader more likely to judge argument errors as changing the main claim? For the last question we use a small severity study. We also repeat the labelling on the MIMIC-CXR error categories of Evaluating Progress in Automatic Chest X-Ray Radiology Report Generation. That data asks whether false findings and wrong locations behave as argument errors while severity wording behaves as modifier errors. This severity question is secondary and exploratory.
The second stage builds the restricted editor. A detector marks error spans, as in FAVA or ZINA. A parser maps each span to its subtree and attachment type. The editor then chooses among three operations. The first excises a modifier subtree and adjusts any leftover punctuation or function words. The second substitutes an argument slot with a phrase from retrieved evidence, and its output is constrained to be a constituent of the same type. The third abstains, either by deleting the clause or by inserting an explicit hedge.
We train this editor on operation-typed synthetic data built from the same seed passages FAVA used. Modifier errors are made by attaching a false modifier subtree beside an existing node. Argument errors are made by substituting a false constituent into a core slot. This gives grammatical corruptions with known gold repairs, generated at the scale of FAVA's 35,000-instance training set.
Baselines
The naive reference points are two. The first is the unedited output, which shows how much any editor helps. The second is "delete every flagged sentence," which inflates factual precision by throwing away content. That second baseline is why every result must be reported alongside a preservation measure. Preservation is the fraction of the original output's verified-true atomic claims that survive the edit, computed with the same fixed decomposer.
The strongest existing method is FAVA, a retrieval-augmented editor trained on synthetic error-insertion data. It reports 48.1% detection F1 on FAVABENCH, against 42.5% for GPT-4 and 27.8% for retrieval-augmented ChatGPT. It also reports that editing improved Alpaca 13B's FActScore on biographies by up to 9.3%. We will rerun released FAVA weights on identical inputs with the same retriever and corpus. To separate detection from repair, both FAVA and our editor will be given the same detected spans. Our editor counts as beating FAVA if it matches FAVA's FActScore gain within noise while retaining significantly more verified claims. It also wins if it gives a higher gain at equal preservation. Precision alone does not count.
The second strong competitor is a frontier model prompted with the taxonomy and retrieved evidence to rewrite the flagged spans. It is given the same detected spans and evidence, and its prompt is tuned on the same development budget as our editor. ZINA reports GPT-4o well below its specialized model in the multimodal setting: detection F1 of 29.37 versus 45.15, and editing BERT-F1 19.1 points lower. No reported result was found for frontier-model rewriting with preservation measured, so we will measure it ourselves.
The key ablation is the same editor, trained on the same synthetic seed passages, but with free rewriting and without operation types. A second ablation keeps operation types but swaps grammar-derived argument/modifier labels for random labels with matched proportions.
Why not just…?
Why not just let a strong model rewrite the flagged spans freely? This is a real threat on precision, and we do not assume it loses there. The argument for the restriction rests on collateral damage. Free rewriting can silently alter correct neighbouring content, and no existing benchmark in the file measures this. The preservation metric settles the question. If the frontier rewriter matches our editor on both precision and preservation, the restriction buys nothing for editing quality. The contribution then reduces to the taxonomy finding plus a cheaper, auditable editor.
Why not just run a dependency parser and delete the subtree under each flagged span? This is essentially our excision operation without the type distinction. The proposal is honest that it is a near neighbour, which is why it appears as the random-type ablation and as a delete-everything-typed variant. The prediction that separates them is specific. Blind deletion should match our editor on modifier errors but damage sentences on argument errors. Deleting a core participant should leave ungrammatical or misleading text, and we will check this with grammaticality judgments and the preservation metric. If blind deletion matches us on argument errors too, the argument/modifier distinction is not doing work, and we will report that.
Why not just decompose into atomic claims and drop the false ones? Dropping atomic claims is not an edit to the text. A writer still has to reconstruct a sentence from the surviving claims, which reintroduces free rewriting. Wanner et al. also show that the claim lists themselves are unstable. Our edits operate on the original text, so the output stays close to what the model wrote. We still use a fixed predicate–argument decomposition for scoring.
Proposed Experiments
The decisive first check is purely analytic. On FAVABENCH, we test whether modifier-attached spans are over-represented among invented and unverifiable errors. We also test whether gold human edits on modifier spans are mostly deletions, while edits on argument spans are mostly substitutions. A null result here, meaning attachment type independent of both content category and gold repair, ends the editing program before any training.
If that check passes, editing is evaluated in two settings. The first is the FActScore biography task, using the generators FAVA reported on and holding the decomposer fixed. The second is FAVABENCH editing. In both, we report factual precision, preservation of verified claims, and grammaticality from human raters on a sample.
The core result is the precision–preservation curve of our editor against FAVA, the prompted frontier rewriter, blind subtree deletion and sentence deletion. Results are broken down by attachment type, so we can see whether gains come where the conjecture predicts. A final robustness check reruns the analysis with parser output degraded on the hand-verified sample. This shows how much of any gain survives realistic parse errors.
References
- Fine-grained Hallucination Detection and Editing for Language Models
- ZINA: Multimodal Fine-grained Hallucination Detection and Editing
- MedCalc-Bench: Evaluating Large Language Models for Medical Calculations
- D-Tree Grammars
- Arguments and Modifiers from the Learner's Perspective
- A Closer Look at Claim Decomposition
- FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
- Machine Translation Testing via Syntactic Tree Pruning
- Evaluating progress in automatic chest X-ray radiology report generation