When a language model answers a question from retrieved tables and text, the hallucinations that are hardest to catch are not invented facts. They are binding errors. The model quotes a real number but attaches it to the wrong company, year, or treatment arm. It swaps a donor's age with the recipient's. It reports 10.5% where the table says 10.2%. These answers look right to any check that compares meanings, because every word in them is plausible and every entity in them appears in the evidence. The strongest current detector for financial answers, FinGround, splits answers into small atomic claims and re-computes the numerical ones. Even so, its recall drops to 71.4% on values within ±5% of the truth.
We propose to check each claim by its roles rather than by its meaning. For every number or attribute in a generated answer, a staged extractor first finds the entity the claim is about. It then finds the attribute and time period, given that entity. Only then does it read off the value, given both. Decoupled (cascaded) relation extraction works this way: it splits relation finding into ordered steps, each conditioned on the one before. The resulting tuple of entity, attribute, period, value and unit is looked up in an index keyed by table coordinates (row entity, column header, period). The value is then compared numerically, or re-derived by executing a short program. Our claim is that this conditional role binding, and not claim decomposition alone, is what catches swaps and near-misses. The experiments are built to show whether that is true.
Core Claim
FinGround already decomposes answers into claims and re-computes formulas. What we change is what the verification unit is. FinGround's atomic claims are free-text sentences checked against retrieved evidence. Ours are typed tuples whose slots are extracted in a fixed order, each conditioned on the previous slot.
There is evidence that conditioning on the head entity and relation is what separates overlapping roles. In PasCore, a three-stage cascade (relation, then head entity, then tail conditioned on both) reached 60.7 F1 on ICRED, against 55.1 for GRTE and 55.0 for CasRel. Removing the relation conditioning from the tail stage dropped it to 44.4. E2CNN uses the same conditional design for financial text. Role swapping is an overlapping-role problem in disguise: one sentence binds several values to several entities. Our hypothesis is that a verifier built on conditional extraction will detect swaps that sentence-level checks pass.
We make two falsifiable predictions. First, the method will raise near-miss recall on FinHalu above FinGround's reported 71.4%. Second, on controlled swap probes, it will beat both semantic checking and plain number matching, and the gain will vanish when the conditioning is removed.
Research Program
The system has three parts.
The first part is a coordinate index built at retrieval time. Every table cell retrieved by the RAG step (retrieval-augmented generation, where evidence is fetched and placed in the prompt) is stored with its row entity, column header, period and unit. Headers come from table structure, so no entity recognizer runs over the tables. This matters because Rethinking Document-Level Relation Extraction showed that relation extractors collapse by over 80% F1 once their entity inputs are noisy. That paper constrains our design. We keep extraction off messy source documents and run it only over the short, well-formed generated answer.
The second part is the cascade itself, run over the answer. Stage one marks claim heads, meaning entities and cohorts. Stage two predicts the attribute and period conditioned on each head. Stage three extracts the value span conditioned on the head and attribute, using conditional layer normalization as in PasCore.
We will build it in two forms. One is a small trained encoder. The other is a prompted LLM that issues three separate calls, one per stage. Training labels come for free from datasets that link answers to cells. FinQA supplies gold reasoning programs that name cells. TAT-QA supplies derivations. We align these to answer text, and use LLM-proposed labels where alignment fails.
The third part is the checker. A tuple that matches a cell coordinate is compared exactly, within the precision printed in the table. A tuple whose value is computed is re-derived with a short program, following the plan-then-execute approach of Mao et al. and TART. A tuple whose coordinates match nothing is labeled "unverifiable" rather than passed.
That last rule is how we handle the cascading-error worry. HybridQA showed that stages each near 88% accurate compound to about 44% end to end. We therefore report extractor recall on answers separately. We also treat extraction misses as abstentions, so they never become silent approvals. Flagged claims are regenerated with cell citations, as in FinGround. Regeneration is not our contribution.
Baselines
The naive reference point is the generator with no verification: GPT-4o over the same retrieved evidence. No reported result was found on Lacuna for a standalone GPT-4o hallucination rate on FinHalu. FinGround reports only that its full pipeline cuts hallucinations by 78% relative to GPT-4o. We will measure this directly.
The strongest existing method is FinGround's claim-level detector. Its distilled 8B model reaches 91.4% F1 on FinHalu, keeping 96.2% of its GPT-4o teacher, with 71.4% recall on ±5% near-misses. FinGround stresses "retrieval-equalized" comparison, and we adopt it: every detector sees identical retrieved chunks. If FinGround's code is unavailable, we will reimplement it from the paper. We will use the same 8B backbone for our trained cascade, the same distillation data budget, and the same prompt-tuning budget for the prompted variants. We will quote the published numbers alongside our reimplementation. Beating this baseline means matching its overall F1 within noise while raising near-miss recall and swap-probe recall by a margin that holds across seeds. A higher overall F1 alone does not count.
For table-only claim verification on TabFact, the strongest method in the file is TIDE, which turns questions into triples and checks answers against them. It reports 89.82% accuracy. We run it with the same GPT-3.5-class backbone.
The ablation that isolates our key ingredient keeps everything identical but extracts all tuple slots in one unconditioned pass. A second ablation replaces the coordinate index with similarity-based matching of tuples to cells.
Why not just…?
Why not just match the numbers in the answer against the numbers in the evidence? This is the most serious threat, because it would catch many near-miss values cheaply. It cannot catch a swap, where the number is real but bound to the wrong row or period. It also cannot handle computed values, which appear nowhere in the evidence, and FinHalu includes computational claims. We include it as a baseline. If it matches our method on the swap probes, the role-binding idea is wrong and we will say so.
Why not just have a strong LLM judge each claim, or ask the generator to cite cells? FinGround's GPT-4o teacher is exactly such a judge, and its students still miss near-misses. TIDE also reports self-correction bias: models defend their own errors when asked to verify them. Our GPT-4o-judge baseline, given identical evidence, will show whether a larger judge closes the gap without structure.
Why not just generate code so the answer is computed rather than written? Programs fix arithmetic, as TART's 41.9% gain over chain-of-thought (step-by-step written reasoning) shows. They do not fix the prose wrapped around the result. A program can return the right value while the answer attributes it to the wrong segment or arm. Code also breaks on messy tables. We report how many hallucinations remain in program-of-thought answers, and how many of those our checker catches.
Proposed Experiments
The main evaluation is detection on FinHalu: 1,200 expert-annotated question-answer-evidence triples. We report claim-level F1, per-type recall across its six claim types, near-miss recall, and latency against FinGround's $0.003 per query. Transfer is tested on TAT-QA, FinQA and FinanceBench answers produced by several open and closed generators.
Following the probing method of the Reality Check paper, we build swap probes by perturbing correct answers. The perturbations are row swaps, column swaps, period swaps, unit changes, and value shifts of 0.5–5%. They are applied to FinQA, TAT-QA and HybridQA answers. They are also applied to NLI4CT clinical trial statements, where swapping intervention and control arms mirrors the patient–donor swaps noted by MEDIMP. Metrics are detection recall per perturbation type, plus false-alarm rate on unperturbed answers.
On TabFact we compare against TIDE's 89.82%, with the complex-statement split reported separately. On HybridQA we check whether gating answers with our verifier improves exact match over S3HQA's reported 67.9 EM without lowering coverage.
Two checks decide the outcome. First, the conditioned cascade must beat the one-pass ablation on swap probes; otherwise conditioning is not the cause of any gain. Second, it must beat number matching on swaps while equaling it on pure value shifts; otherwise the added machinery is not worth its cost. We also report extractor recall on answers and the abstention rate, so that coverage cannot hide misses.
References
- FinGround: Detecting and Grounding Financial Hallucinations via Atomic Claim Verification
- PasCore: A Chinese Overlapping Relation Extraction Model Based on Global Pointer Annotation Strategy
- E2CNN: entity-type-enriched cascaded neural network for Chinese financial relation extraction
- Rethinking Document-Level Relation Extraction: A Reality Check
- FinQA: A Dataset of Numerical Reasoning over Financial Data
- PoTable: Programming Standardly on Table-based Reasoning Like a Human Analyst
- TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning
- HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data
- Triples as the Key: Structuring Makes Decomposition and Verification Easier in LLM-based TableQA
- Saama Technologies at SemEval-2024 Task 2: Three-module System for NLI4CT Enhanced by LLM-generated Intermediate Labels
- MEDIMP: 3D Medical Images and clinical Prompts for renal transplant representation learning
- S3HQA: A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answering