Listener-Augmented Thinking VLMs for Robust Text-to-Image Alignment

Alexander Gambashidze, Pengyi Li, Matvey Skripkin, Konstantin Sobolev, Andrey V. Galichin, Andrey Kuznetsov, Ivan OseledetsOpenReview 2026 · Submission to ICLR 2026 · Withdrawn · OpenReview

Its authors withdrew this paper, so there is no full text to read or summarize.

Abstract

Improving the generalization and robustness of reward models for visual preferences is crucial-especially under limited annotated data, where collection is costly and complex. While we show that reinforcement learning with verifiable rewards (RLVR) improves generalization, we identify a key failure mode: models often produce final answers that are inconsistent with their own chain-of-thought (CoT); when the same trace is shown to a frozen vision–language model (VLM), it frequently predicts a different choice, and accuracy drops as this inconsistency grows. To solve this, we propose a listener-augmented GRPO objective: a frozen listener re-evaluates the reasoner’s CoT (with the final answer masked) and provides a dense soft reward, combined with an exact-match term for unbiased supervision. We further introduce a probability-based pairwise scoring scheme: at training time it yields calibrated confidence rewards for GRPO, and at inference time it supports an anchor-based strategy that can avoid $\mathcal{O}(n^2)$ comparisons while still producing reliable win probabilities. Under this framework, we establish a clear hierarchy - \textbf{single-image scoring $<$ pairwise comparison $<$ RL with CoT $<$ RL with CoT + listener} - showing that conditioning on both images already boosts generalization, CoT improves further, and listener shaping provides the largest gains. Despite training on only 16\% of HPDv2, our approach achieves 67.4\% on the ImageReward test set and improves OOD accuracy by up to +6 pp on a modern 1.2M-vote benchmark, while reducing contradiction rates. Integrated into a Flow-GRPO text-to-image pipeline and evaluated with the latest HPSv3 metric on prompts from HPDv2 and T2I-CompBench, it yields consistent gains. We will release code and models to facilitate reproducibility.
BibTeX
@misc{gambashidze2026listener,
  author = {Alexander Gambashidze and Pengyi Li and Matvey Skripkin and Konstantin Sobolev and Andrey V. Galichin and Andrey Kuznetsov and Ivan Oseledets},
  title = {{Listener-Augmented Thinking VLMs for Robust Text-to-Image Alignment}},
  year = {2026},
  url = {https://openreview.net/forum?id=w7hGj6pT8d},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2026},
}

Loading related content...

Figure