FiReIQ: Field-Level Confidence Estimation with Reasoning and Image Quality in Document Extraction

Xiao Liu, Yage Wang, Qi Yu, Giri IyengararXiv 2026 · arXiv

TL;DR

FiReIQ is a new framework that provides reliable, field-specific confidence scores for document extraction by fusing a VLM’s internal reasoning traces with granular image-quality metrics. By knowing exactly when to trust an extraction and when to flag it for human review, it significantly boosts automation rates in high-stakes financial and legal workflows without requiring expensive repeated sampling.

What Problem Does This Paper Address?

Vision-Language Models (VLMs) like GPT-4o or Gemini are incredibly good at "reading" documents, but they have a major flaw: they are often overconfident, even when they are wrong. In "high-stakes" industries like finance or law, a single misplaced decimal point in a receipt or a misidentified party in a contract can have massive downstream consequences.

Current uncertainty estimation methods suffer from three main issues:

  1. Coarseness: They usually provide a single confidence score for an entire page. However, a model might correctly extract a "Total Price" but hallucinate the "Vendor Name" on the same document.
  2. Visual Blindness: Models often ignore physical degradations like blur, noise, or low contrast, leading to "confident hallucinations" where the model guesses based on context rather than evidence.
  3. Efficiency: Many reliable methods require "shuffling" the model multiple times (repeated sampling) or running external OCR engines, which is slow and expensive.

The authors from Capital One introduce FiReIQ to solve these by attributing uncertainty to individual fields in a single pass.

Methods

FiReIQ operates on a "single-pass" architecture using structured decoding (via vLLM) to extract values, reasoning excerpts, and token probabilities simultaneously. The framework consists of two main branches:

1. The Reasoning Branch

Instead of just looking at the final answer, FiReIQ analyzes the model's "thought process":

  • Probability Features: It tracks token-level uncertainty (negative log probabilities) through a Gated Recurrent Unit (GRU) to capture patterns of "hesitation" during generation.
  • Semantic Features: It uses a text encoder (BGE-M3) to embed the verbatim reasoning text. If a model writes, "the 5 is faint," the semantic embedding captures that linguistic cue of uncertainty.

2. The Quality-Aware Visual Branch

This is where FiReIQ shines. It doesn't just look at the image; it assesses its "reliability":

  • Spatial Quality Grid: The image is divided into an 8x8 grid. For each tile, the system calculates six metrics: edge acutance, frequency-domain sharpness, entropy, perceptual blur, contrast, and noise.
  • FiLM (Feature-wise Linear Modulation): Rather than just concatenating these quality scores, FiReIQ uses them to modulate the visual features from a SigLIP2 encoder. Essentially, the quality metrics tell the model which parts of the visual representation to "turn up" or "turn down" based on how clear the evidence is.

Finally, these features are fused and passed through a Multi-Layer Perceptron (MLP) to produce a calibrated probability of correctness for every single field.

Datasets

The authors introduced FiReIQ-BENCH, a new benchmark for field-level confidence, built upon two existing datasets:

  • CORD v2: Real-world receipts (focusing on 24 transaction fields).
  • CUAD: Complex legal contracts (focusing on 8 critical entities like "Governing Law" or "Expiration Date").

They also validated the system on private financial service data, representing actual production-scale workflows. The experiments utilized two powerful open-weights models: Gemma 26B and Nemotron 3.5 Super VL 120B.

Key Insights and Ideas

  • Reasoning is a Gold Mine: The paper confirms that a model’s reasoning trajectory contains far richer uncertainty signals than the final answer alone. Field-specific reasoning reduced the error metric (AURC) by over 50% compared to looking at the full generation trajectory.
  • Quality as a Modulator, Not Just a Feature: The use of FiLM to let image quality "condition" the visual features outperformed simple concatenation. This suggests that image quality acts as a "gatekeeper" for how much the model should rely on visual evidence.
  • Local Matters: Global image quality scores are insufficient. By using a spatial grid, FiReIQ can identify that a document is mostly clear but has a specific coffee stain or blur over the "Total Amount" field.

Key Results

FiReIQ consistently outperformed standard baselines (like Sample Agreement or basic LogProbs) across all metrics:

  • Automation Gains: On private financial data, FiReIQ improved coverage at 95% precision by 10.6 percentage points over existing baselines. This translates directly to thousands of fewer documents requiring manual human review.
  • Discrimination: It improved AUROC (Area Under the Receiver Operating Characteristic) by 6.8% on private data, showing a superior ability to distinguish correct extractions from errors.
  • Robustness: On the challenging CUAD (legal) dataset, FiReIQ reduced the Brier score (a measure of probabilistic accuracy) by up to 16.2% compared to multimodal models that didn't explicitly account for image quality.

Limitations and Future Work

  • Verbatim Extraction Failures: The method relies on the VLM being able to provide "verbatim" excerpts of its reasoning. If a model paraphrases its reasoning too much, it becomes difficult to align token probabilities, leading to some data loss (about 6-17% in their tests).
  • VLM Dependency: The system's performance is inherently tied to the quality of the underlying VLM's reasoning. If a model doesn't "know" it's confused, the reasoning trace won't help.
  • Future Directions: The authors suggest exploring even tighter integration with the VLM’s internal hidden states and extending the framework to handle multi-page documents where context might be spread across several images.

Summary updated

Abstract

Vision-Language Models (VLMs) have demonstrated strong performance in structured document extraction, yet their deployment in high-stakes applications requires reliable confidence estimates for individual fields. Existing uncertainty estimators often aggregate signals across an entire generation, obscuring field-specific reasoning uncertainty while overlooking visual degradations that may lead to confident extraction errors. To address these limitations, we introduce FiReIQ, a field-level confidence estimation framework that combines field-aligned reasoning signals from a single VLM generation with local and global image-quality features through quality-conditioned visual fusion. To facilitate systematic evaluation, we further introduce FiReIQ-Bench, a benchmark spanning financial receipts and legal contracts. Experiments with two reasoning VLMs show that FiReIQ consistently improves confidence discrimination, selective prediction, and probabilistic accuracy over existing estimators. These gains extend to private financial service data, where FiReIQ improves AUROC by 6.8% and coverage at 95% precision by 10.6% over an existing baseline. Together, these results highlight the potential of FiReIQ to reduce manual review while enabling more reliable document automation.
BibTeX
@misc{liu2026fireiq,
  author = {Xiao Liu and Yage Wang and Qi Yu and Giri Iyengar},
  title = {{FiReIQ: Field-Level Confidence Estimation with Reasoning and Image Quality in Document Extraction}},
  year = {2026},
  eprint = {2611.18865},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18865},
}
Figure