Listener-Augmented Thinking VLMs for Robust Text-to-Image Alignment
Its authors withdrew this paper, so there is no full text to read or summarize.
Lists
Only you see this
Abstract
Improving the generalization and robustness of reward models for visual preferences is crucial-especially under limited annotated data, where collection is costly and complex. While we show that reinforcement learning with verifiable rewards (RLVR) improves generalization, we identify a key failure mode: models often produce final answers that are inconsistent with their own chain-of-thought (CoT); when the same trace is shown to a frozen vision–language model (VLM), it frequently predicts a different choice, and accuracy drops as this inconsistency grows. To solve this, we propose a listener-augmented GRPO objective: a frozen listener re-evaluates the reasoner’s CoT (with the final answer masked) and provides a dense soft reward, combined with an exact-match term for unbiased supervision. We further introduce a probability-based pairwise scoring scheme: at training time it yields calibrated confidence rewards for GRPO, and at inference time it supports an anchor-based strategy that can avoid $\mathcal{O}(n^2)$ comparisons while still producing reliable win probabilities. Under this framework, we establish a clear hierarchy - \textbf{single-image scoring $<$ pairwise comparison $<$ RL with CoT $<$ RL with CoT + listener} - showing that conditioning on both images already boosts generalization, CoT improves further, and listener shaping provides the largest gains. Despite training on only 16\% of HPDv2, our approach achieves 67.4\% on the ImageReward test set and improves OOD accuracy by up to +6 pp on a modern 1.2M-vote benchmark, while reducing contradiction rates. Integrated into a Flow-GRPO text-to-image pipeline and evaluated with the latest HPSv3 metric on prompts from HPDv2 and T2I-CompBench, it yields consistent gains. We will release code and models to facilitate reproducibility.
BibTeX
@misc{gambashidze2026listener,
author = {Alexander Gambashidze and Pengyi Li and Matvey Skripkin and Konstantin Sobolev and Andrey V. Galichin and Andrey Kuznetsov and Ivan Oseledets},
title = {{Listener-Augmented Thinking VLMs for Robust Text-to-Image Alignment}},
year = {2026},
url = {https://openreview.net/forum?id=w7hGj6pT8d},
howpublished = {OpenReview},
note = {Submitted to ICLR 2026},
}Loading related content...