TL;DR
This paper introduces DeFacto, a novel framework for multimodal reasoning that tackles the problem of "unfaithful" reasoning in vision-language models (VLMs). DeFacto uses counterfactual examples and a specialized reinforcement learning approach to ensure that models not only provide correct answers but also ground their reasoning in relevant visual evidence, leading to more robust and interpretable AI.
What Problem Does This Paper Address?
Modern multimodal language models (MLLMs), especially those incorporating "thinking with images," have shown impressive capabilities in tasks like visual question answering. However, a critical flaw persists: these models often arrive at correct answers by relying on spurious correlations or irrelevant visual cues, rather than genuine understanding of the image content. This "unfaithful" reasoning can manifest in several ways: the model might point to the wrong evidence for a correct answer, or even get the right answer for the wrong reasons. This lack of evidence-answer consistency hinders interpretability and trust in these AI systems, especially in complex scenarios requiring precise visual grounding. DeFacto aims to bridge this gap by explicitly enforcing that the reasoning process is directly supported by the visual evidence presented in the image.
Methods
DeFacto employs a multi-pronged approach:
- Counterfactual Reasoning Framework: The core idea is to train models using three types of training instances:
- Positive: Standard training with the original image and correct answer.
- Counterfactual: Key evidence regions are masked, forcing the model to abstain (e.g., output "Unknown") if it cannot answer without the evidence.
- Random-Masking: Irrelevant regions are masked to prevent the model from learning shortcuts based on mask patterns alone.
- Automated Evidence Construction Pipeline: To create these counterfactual scenarios without manual annotation, DeFacto uses a pipeline that:
- Descriptor Extraction: A MLLM parses the question to identify key textual descriptors (e.g., "red cup").
- Evidence Localization: An open-vocabulary detector (DINO-X) and an OCR module are used to find image regions corresponding to these descriptors. These are designated as positive evidence regions ($R^+$).
- Instance Generation: Based on $R^+$, the positive, counterfactual ($I \setminus R^+$), and random-masking ($I \setminus R^-$) instances are generated.
- GRPO-based Reinforcement Learning: The trained models are further optimized using Proximal Policy Optimization (PPO) with Group Relative Policy Optimization (GRPO). This is coupled with a tailored reward function that includes:
- Answer Correctness Reward: Encourages correct answers in positive/random cases and "Unknown" in counterfactual cases.
- Format Consistency Reward: Ensures outputs adhere to a structured format (bounding boxes, thinking process, answer).
- Region Selection Coherence Reward: Promotes overlap with evidence regions ($R^+$) and penalizes overlap with irrelevant regions ($R^-$).
Datasets
- DeFacto-100K: A newly constructed dataset of approximately 100,000 images. This dataset is built using the automated pipeline described above and is designed to provide positive, counterfactual, and random-masking instances for training. It leverages existing VQA and document understanding benchmarks.
- DeFacto-1.5K: A human-annotated benchmark consisting of 1,500 samples. This dataset is specifically designed for systematically evaluating region-level reasoning faithfulness beyond just answer accuracy, providing ground-truth evidence annotations.
Key Insights and Ideas
- Counterfactuals are Crucial for Faithfulness: The paper strongly suggests that simply training on correct answers isn't enough. Introducing counterfactual scenarios where critical evidence is removed is essential for teaching models to rely on actual visual support and to know when to abstain.
- Automated Data Generation is Key: The success of DeFacto hinges on its ability to automatically generate counterfactual data. This bypasses the costly manual annotation process, making the approach scalable.
- Joint Optimization is Necessary: Achieving both accurate answers and faithful reasoning requires a holistic approach. The combination of counterfactual supervision and a carefully designed reinforcement learning reward function is shown to be more effective than optimizing for accuracy alone.
- Beyond Accuracy: The paper highlights the limitations of traditional accuracy metrics. DeFacto-1.5K and the analysis of failure modes demonstrate the importance of evaluating how a model arrives at an answer, not just if the answer is correct.
Key Results
- Improved Accuracy: DeFacto consistently improves answer accuracy across various general VQA, document VQA, and scene text-centric benchmarks. For example, on VQAv2, it achieves 72.1% accuracy, a +5.1% improvement over the Qwen2.5-VL baseline.
- Enhanced Faithfulness: DeFacto significantly reduces "Mislocalized & Wrong" and "Spurious Correct" failure cases compared to baselines like GRIT. On the DeFacto-1.5K benchmark, it achieves the best overall scores for faithfulness metrics like mAP, AP50, and AP75.
- Robustness to Masked Evidence: When human-annotated evidence is masked, DeFacto shows significantly higher abstention accuracy, indicating its reasoning is genuinely tied to the visual input.
- Competitive Performance: DeFacto achieves competitive performance against leading closed-source models and state-of-the-art "thinking with images" approaches.
Limitations and Future Work
- Reliance on External Detectors: The evidence localization pipeline relies on existing open-vocabulary detectors. While effective, occasional errors from these detectors can impact the counterfactual data generation. The authors suggest that improvements in detector technology will further enhance DeFacto.
- Dataset Scale: While DeFacto-100K is substantial, it's noted as modest compared to large-scale pre-training corpora.
- Static Images: The current work focuses on static images. The authors identify extending DeFacto to videos and temporal reasoning as a promising future direction.
- Generalizability: The paper suggests that the principles of counterfactual supervision and region-level reward design can be applied to a broader range of multimodal reasoning systems and embodied AI applications.
Summary updated