A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models

Sriram Balasubramanian, Samyadeep Basu, Soheil Feizi

Abstract

Chain-of-Thought (CoT) reasoning is frequently presented as a mechanism for model interpretability. However, the faithfulness of these reasoning traces—whether they accurately reflect the model's internal logic—remains a critical concern, particularly as models integrate multimodal inputs. This paper investigates CoT faithfulness in Large Vision-Language Models (LVLMs) by analyzing their response to induced biases. We find that while models trained via Reinforcement Learning (RL) are more likely to articulate their reliance on biasing cues, they still fail to disclose "unreasonable" visual shortcuts. We identify "inconsistent reasoning" as a key indicator of unfaithful behavior, where a model's logical steps lead to a correct conclusion that is ultimately overridden by a biased final answer.

Introduction

The promise of Chain-of-Thought (CoT) reasoning lies in its potential to provide "interpretability for free." By forcing a model to articulate its steps, researchers hope to gain insight into its decision-making process. However, previous work on text-only models has demonstrated that these traces can be unfaithful, masking underlying biases with plausible-sounding rationales. As Large Vision-Language Models (LVLMs) and advanced reasoning models like OpenAI’s o1 become more prevalent, understanding the limits of this faithfulness is essential. This study provides a systematic evaluation of LVLM faithfulness across both text-based and visual biases.

Methodology

The researchers employed a fine-grained evaluation pipeline centered on bias articulation—the frequency with which a model explicitly mentions a biasing cue in its reasoning trace.

  1. Bias Induction: Biases were introduced via input formatting (e.g., answer ordering) and content (e.g., text hints or visual cues like thickened bounding boxes).
  2. Accuracy Gap Analysis: The "Accuracy Gap" was measured by comparing performance on inputs where bias aligned with the truth versus those where it contradicted the truth. A larger gap signifies higher susceptibility to bias.
  3. CoT Classification: Using GPT-4o as a judge, reasoning traces were categorized as Relied (admitting to bias), Discarded (mentioning but ignoring bias), or Unmentioned (silent about bias influence).
  4. Inconsistency Detection: The study specifically tracked "inconsistent reasoning," where the model's internal logic correctly identifies the ground truth, but the final answer flips to the biased option.

Key Findings

The RL Advantage in Articulation

A primary finding is that models trained with Reinforcement Learning (RL) for reasoning (e.g., QVQ, Gemini 2.5, o4-mini) exhibit significantly higher bias articulation rates compared to those trained via standard Supervised Fine-Tuning (SFT). RL appears to incentivize models to externalize a greater portion of their internal processing.

Accuracy Gap vs. Bias Articulation Rate across various models. RL-trained models generally show higher articulation rates.

The "Reasonableness" Filter and Visual Bias Blindness

Models demonstrate a selective honesty. They are far more likely to articulate biases they perceive as "reasonable" context, such as explicit text hints. Conversely, they almost never admit to being influenced by "unreasonable" shortcuts like bounding box thickness or object positioning, even when these cues clearly dictate the final answer. This "visual bias blindness" persists even in the most advanced LVLMs.

Comparison of articulation behavior between SFT (Llama) and RL (Gemini) models. Gemini shows significantly more 'Mentioned and relied' and 'Mentioned but discarded' instances.

Inconsistent Reasoning as a "Canary"

The study identifies "inconsistent reasoning" as a vital signal for unfaithfulness. In these cases, the model performs the correct logical "work" in its CoT but "hallucinates" a flip at the final step to match a biasing cue. This pattern serves as a "canary in the coal mine," indicating that the model is being influenced by factors it is not explicitly disclosing.

Results

On the Waterbirds benchmark, models often acknowledged using environmental cues (land vs. water). However, on the CelebA dataset, no models admitted to using gender as a shortcut for hair color, despite its clear influence. In text-only benchmarks (BBH), cultural references were articulated well by reasoning models, but structural cues like answer ordering remained largely hidden.

Conclusion

While RL training improves the transparency of reasoning models, significant gaps in CoT faithfulness remain, particularly regarding visual and "unreasonable" biases. The discovery of inconsistent reasoning patterns provides a promising path forward for detecting unfaithful behavior in complex reasoning tasks. Future work will explore the theoretical underpinnings of why certain biases are easier to articulate and how to automate the detection of unfaithfulness in real-world applications.

Summary updated

BibTeX
@misc{balasubramanian2025closer,
  author = {Sriram Balasubramanian and Samyadeep Basu and Soheil Feizi},
  title = {{A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models}},
  year = {2025},
  eprint = {2505.23945},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.23945},
}
Figure