Chain-of-Description: What I can understand, I can put into words

Jiaxin Guo, Daimeng Wei, Zongyao Li, Hengchao Shang, Yuanchang Luo, Hao YangarXiv 2025 · DOI · arXiv

Abstract

Chain-of-Description (CoD) is a novel prompting strategy for Multi-Modal Large Language Models (MLLMs) that requires the model to generate a detailed textual description of an audio or visual input before answering a query. By forcing the model to "put what it understands into words" first, researchers achieved significant performance gains—up to a 5.3% improvement on hard-level vision tasks—demonstrating that explicit verbalization acts as a powerful bridge between raw sensory data and logical reasoning.

Introduction

While Multi-Modal Large Language Models (MLLMs) have become increasingly capable of processing images and audio, they often struggle with a "leap of logic." Standard prompting typically asks these models to jump directly from a complex multi-modal signal—such as a dense scientific diagram or a long audio clip—to a final answer. This direct approach often fails because the model may perceive the necessary details but fail to attend to them during the reasoning phase.

The Chain-of-Description (CoD) Method

To address this, the authors introduce Chain-of-Description (CoD) Prompting, a two-step inference process:

  1. Description Generation: The model is first prompted to provide a comprehensive textual representation of the input (e.g., "Please describe this image in detail").
  2. Query Response: The model then uses its own generated description as context to answer the specific user question.

This approach is conceptually similar to "Chain-of-Thought" (CoT) prompting. However, instead of decomposing a logical problem into steps, CoD decomposes the perception of the input into a structured textual foundation.

Figure 2: Comparison between Standard Prompting and Chain-of-Description Prompting on a microscopic image task. CoD helps the model identify key features like the cell wall and nucleus to reach the correct answer.

Experimental Setup and Datasets

The study evaluated CoD across two primary domains:

  • Audio (LALMs): Evaluated using AIR-Bench-Chat, containing 2,200 samples across Speech, Sound, Music, and Mixed Audio.
  • Vision (LVLMs): Evaluated using MMMU_Pro, a robust benchmark categorizing tasks into Easy, Medium, and Hard levels.

Key Insights

The core philosophy of CoD is that "what I can understand, I can put into words." The generated description serves as a sanity check for the model's internal state.

Interestingly, CoD acts as a "Hard Task" Specialist. While the detailed description can sometimes introduce "noise" for easy tasks, it provides essential "anchors" for complex reasoning. Furthermore, the researchers found that CoD is particularly effective for human speech due to its high information density—rich in textual content, emotions, and context.

Results

Quantitative results demonstrate the effectiveness of CoD across various models and difficulty levels. On the MMMU_Pro benchmark, CoD increased the accuracy of both Qwen2-VL and Qwen2.5-VL on "Hard" level tasks by 5.3%.

In the audio domain, using Qwen2-Audio, CoD improved speech category performance by nearly 4% on the AIR-Bench-Chat benchmark, with an average alignment increase of 1.79% across all categories.

Conclusion and Future Work

While CoD significantly improves reasoning accuracy, it introduces computational overhead due to increased latency and token consumption. Future work may involve integrating "description-first" tasks into the pre-training phase of MLLMs to bake this capability directly into the model's weights, potentially mitigating the inference-time cost.

Summary updated

BibTeX
@misc{guo2025chain,
  author = {Jiaxin Guo and Daimeng Wei and Zongyao Li and Hengchao Shang and Yuanchang Luo and Hao Yang},
  title = {{Chain-of-Description: What I can understand, I can put into words}},
  year = {2025},
  eprint = {2502.16137},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.16137},
}
Figure