TL;DR
Large Vision-Language Models (LVLMs) frequently omit or misrepresent fine-grained details when generating image captions. This paper introduces Cross-modal Identity Mapping (CIM), an annotation-free reinforcement learning framework that uses text-to-image retrieval consistency as a proxy reward to train models to generate highly precise and detailed captions.
What Problem Does This Paper Address?
While modern LVLMs excel at general visual understanding, they suffer from significant "information loss" during image-to-text conversion. They tend to describe coarse concepts (e.g., "a dog") while ignoring or hallucinating fine-grained details (e.g., specific breeds, colors, or spatial relations).
Fixing this typically requires Supervised Fine-Tuning (SFT) on massive, meticulously labeled datasets, which are expensive and time-consuming to build. Alternatively, existing reinforcement learning (RL) approaches use other Vision-Language Models to score captions, but these are notorious for "reward hacking" because the evaluator models themselves struggle with complex compositional reasoning.
The authors tackle a fundamental question: How can we quantitatively measure and minimize information loss in image captioning without relying on human annotations or flawed VLM evaluators?
Methods
The core insight of CIM is that the quality of an image caption is positively correlated with the uniqueness of the images it can retrieve.
Instead of directly comparing text to an image, CIM converts the problem into an image-to-image similarity task using a text-retrieval loop:
- The LVLM generates a caption for a source image.
- This caption is used as a query to retrieve the top-$K$ most relevant images from a large gallery.
- The framework evaluates the caption based on two novel, annotation-free metrics:
- Gallery Representation Consistency (GRC): Measures how visually similar the retrieved images are to each other. If a caption is highly detailed, the retrieved gallery will be tightly clustered and consistent.
- Query-gallery Image Relevance (QIR): Measures how visually similar the retrieved images are to the original source image, applying an exponential decay weight based on retrieval rank. This ensures the caption is accurate and not just detailed but wrong.
These two metrics are combined into a single reward signal to optimize the LVLM using Group Relative Policy Optimization (GRPO), forcing the model to minimize information loss and achieve an "identity mapping" from image to text.
Datasets
- Training & RL Optimization: RefinedCaps (6.5K images sampled from the COCO training split).
- Retrieval Corpus: RefinedCaps augmented with DenseFusion-1M to provide a large pool of candidate images for the text-search mechanism.
- Evaluation Benchmarks:
- DOCCI500: A 500-image subset of the Detailed Object Description dataset, focusing on long, non-human-centric descriptions.
- COCO-LN500: A 500-image subset from Localized-Narratives, filtered specifically for complex, human-related descriptions.
- Verification: Oxford-IIIT Pet dataset was utilized to verify the initial hypothesis that coarse vs. fine-grained classification errors correlate directly with information loss.
Key Insights and Ideas
- Bypassing the Modality Gap: By evaluating caption quality through the lens of image-to-image similarity in a retrieved gallery, the authors elegantly sidestep the difficulty of directly aligning text embeddings with visual embeddings.
- The "Detailed but Wrong" Trap: As shown in Figure 1, a caption can be rich in detail but factually incorrect. CIM's dual-metric reward ensures that a model cannot cheat by simply generating verbose, highly specific text (which would score high on GRC) if it doesn't match the source image (which drops the QIR score).
- Exploiting Corpus Scale: The authors discovered that as the retrieval database grows, the number of "hard negative" images increases. This naturally raises the bar for the model, forcing it to generate even more precise captions to successfully retrieve the correct visual cluster.
Key Results
The CIM framework was tested across multiple prominent open-source backbones (LLaVA1.5, Qwen2-VL, Qwen2.5-VL, and InternVL) and demonstrated substantial improvements:
- Massive Boost in Reasoning: On the COCO-LN500 benchmark, CIM achieved a 20.4% absolute improvement in relation reasoning (QA score) when applied to Qwen2.5-VL-7B.
- SFT Enhancement: When applied on top of already strong Supervised Fine-Tuning baselines, CIM consistently pushed performance further, outperforming alternative RL methods like SC-Captioner.
- Slaying the Giants: Remarkably, 7B and 8B parameter models fine-tuned with CIM frequently outperformed their massive 72B+ parameter base models across overall caption quality metrics (CAPTURE) and attribute precision.
- Encoder Agnosticism: Robustness checks showed that CIM is highly stable; changing the underlying image or text encoders (e.g., swapping OpenCLIP for DINOv3) resulted in negligible performance variance (<0.8%).
Limitations and Future Work
While CIM significantly boosts descriptive granularity and relationship reasoning, the authors noted a minor trade-off: the model sometimes suffers a slight drop in raw object precision compared to the base models. This suggests that while global-level rewards successfully encourage detail and structure, they can occasionally gloss over fine-grained object recognition nuances.
Future work will likely focus on introducing localized or object-level constraints into the retrieval reward loop to mitigate this minor trade-off.
Summary updated