TL;DR
MLIE-CoT is a visual "Chain-of-Thought" framework that improves text-to-image generation by iteratively diagnosing and repairing images at multiple levels of granularity. By training a specialized router to choose between local edits, global refinements, or full regenerations, the system preserves correct visual details while systematically fixing compositional errors, outperforming state-of-the-art models like FLUX.2 on complex benchmarks.
What Problem Does This Paper Address?
Despite the impressive quality of modern text-to-image (T2I) models, they frequently stumble on complex, compositional prompts. Common failures include missing objects, incorrect attribute binding (e.g., a "red hat and blue shirt" becoming a "blue hat and red shirt"), and reversed spatial relationships.
Existing "Chain-of-Thought" (CoT) approaches for images typically rely on textual reasoning or repeated whole-image regeneration. However, these methods face two major hurdles:
- Negative Visual Anchors: If an initial image has a fundamentally broken layout, using it as a condition for the next step often carries those errors forward, leading to accumulated hallucinations.
- Collateral Damage: Attempting to fix a small detail via whole-image editing often inadvertently changes parts of the image that were already correct.
Methods
The authors propose MLIE-CoT (Multi-Level Image Editing Chain-of-Thought), a framework that treats image generation as a test-time control process. The pipeline follows a loop: Analyze → Route → Plan → Edit → Review.
The core innovation is the Multi-Level Repair strategy, which allows the model to choose the "intervention strength" based on the severity of the error:
- L1 (Local Repair): Uses bounding boxes and crops to focus the editor on specific regions (e.g., changing a pizza's color). This minimizes disruption to the rest of the scene.
- L2 (Whole-Image Editing): Adjusts global attributes like background or spatial relations while keeping the general layout.
- L3 (Text-Only Regeneration): Discards the current image entirely and starts fresh if the current state is a "negative anchor" that would mislead further edits.
To make this work, they trained a Repair-Level Router using a frozen Qwen3.5-9B multimodal backbone. Unlike simple classifiers, this router is trained using Utility-Supervised Learning, where it learns to predict which repair level will yield the highest "utility"—a score balancing error correction, content preservation, and computational cost.
Datasets
- Training Dataset: The authors constructed a custom image-repair dataset consisting of 14.8K images derived from 1K prompt families. They intentionally created "controlled errors" by perturbing prompts to generate mismatched images, providing clear ground truth for what needs fixing.
- Evaluation Benchmarks:
- GenEval: A benchmark for compositional consistency (objects, counts, colors, positions).
- WISE: A challenging dataset requiring implicit knowledge (e.g., generating an image based on a scientific or cultural clue rather than a direct description).
Key Insights and Ideas
- Granularity is the Key to Preservation: The "aha moment" here is that the model shouldn't always look at the whole image. By using local crops (L1), the model can fix a single object's color without changing the background, a common failure in standard diffusion editing.
- Routing vs. LLM Prompting: Simply asking an LLM to "fix the image" is often unstable. By training a router on actual utility scores (how much an action actually improved the image in practice), the system becomes much more reliable at choosing the right tool for the job.
- Visual States as Reasoning: In MLIE-CoT, the intermediate images are not just outputs; they are the "thoughts" in the chain. The framework treats these images as reusable states that can be accepted, modified, or rejected.
Key Results
MLIE-CoT demonstrated significant gains across multiple base models:
- GenEval: Improved BAGEL from 0.80 to 0.86 and FLUX.2-dev from 0.84 to 0.88 in overall accuracy. The gains were most prominent in difficult categories like spatial positioning and attribute binding.
- WISE: Raised the overall score from 0.70 to 0.76 (BAGEL) and 0.73 to 0.78 (FLUX.2).
- Efficiency: Ablation studies showed that 1–3 repair steps offer the best trade-off between quality and cost. Beyond three steps, the gains tend to plateau as the model hits the inherent capacity limits of the underlying generator.
Limitations and Future Work
- Resolution Constraints: The current framework operates at fixed resolutions (e.g., 1024x1024). Extending this to higher resolutions or variable aspect ratios remains a challenge.
- Prompt Optimization: While the router selects the level of repair, the specific textual instructions sent to the editor could be further optimized to reduce ambiguity.
- Computational Overhead: Although more efficient than exhaustive search, iterative editing still requires multiple forward passes, which may be too slow for real-time applications without further optimization.
Summary updated