TL;DR
MANGO (Manga Active Narrative Grounding Optimization) is an unsupervised reinforcement learning framework that transforms Manga Visual Question Answering from a passive full-page reading task into an active, question-guided panel exploration. By iteratively building a compact "evidence sketch" and optimizing it via group-relative policy training with self-ranking and path-consistency rewards, MANGO achieves state-of-the-art performance without requiring human-annotated answers or rationale paths.
What Problem Does This Paper Address?
Reading a manga page isn't like looking at a standard natural image; it is a highly structured multimodal narrative. Information is non-linearly distributed across ordered panels, embedded dialogue bubbles, recurring characters, and implicit transitions between events.
Standard Multimodal Large Language Models (MLLMs) typically process an entire manga page in a single passive pass. This often causes them to miss fine-grained details, hallucinate connections, or generate overly verbose, redundant answers. While "active perception" (allowing a model to dynamically zoom or crop into regions) can help, training models to navigate these complex layouts usually requires expensive, human-annotated step-by-step reasoning paths (rationales), which are incredibly difficult to scale.
Methods
The authors introduce MANGO, a framework that reframes manga VQA as an iterative, question-driven evidence-seeking process. It couples two core components:
- Active Narrative Sketching (ANS): Instead of reading panels in a fixed sequence, the model dynamically decides which panel to inspect next based on the question and its current knowledge state. At each step, it performs three actions: selects a panel ID, extracts a concise, grounded text clue, and decides whether to continue exploring or stop and answer. This builds a compact "evidence sketch."
- Unsupervised Reward Modeling & GRPO: To train this behavior without gold-standard human answers, MANGO uses Group-Relative Policy Optimization (GRPO)—a critic-free reinforcement learning variant—driven by two unsupervised rewards:
- Answer Preference Reward ($R_{pref}$): The model samples multiple reasoning paths (rollouts), clusters identical final answers, and uses the policy model itself to rank the unique candidates (listwise self-ranking).
- Path Consistency Reward ($R_{cons}$): It organizes the panel trajectories of high-ranking rollouts into a prefix tree, rewarding paths that consistently follow stable, reliable sequences of crucial panels.
Datasets
- Training: A question-only subset of 512 questions from MangaVQA, utilizing only the questions and Manga109 panel bounding box annotations. Crucially, no ground-truth answers or human rationales were used to compute training rewards.
- Evaluation: Evaluated across three major benchmarks:
- MangaVQA: Tests open-ended exact extraction and descriptive answering.
- ChrOMIC: Evaluates chronological reasoning across multi-panel comics (Japanese manga subset) in both multiple-choice and open-ended formats.
- MangaUB: A large-scale benchmark probing single-panel evidence extraction and multi-panel evidence aggregation.
Key Insights and Ideas
- Question-Directed Order vs. Reading Order: The optimal path to answer a specific question often deviates from the standard panel reading order (e.g., jumping ahead to see an outcome, then back to identify a speaker). ANS allows the model to dynamically restructure the narrative flow based on informational need.
- Conciseness Over Verbosity: Unlike other active perception methods that accumulate lengthy visual descriptions, MANGO forces the model to extract atomic, local clues. This structured brevity prevents the model from getting lost in redundant data, leading to higher final accuracy.
- Self-Supervised Rationales via Consistency: By rewarding trajectories that form stable prefixes in a tree structure, the framework successfully discovers true causal paths across panels entirely through self-supervision.
Key Results
- State-of-the-Art Performance: Built on a Qwen3-VL-8B backbone, MANGO consistently outperformed both general perception MLLMs and existing active perception baselines. On MangaVQA, it achieved an overall score of 6.90, outperforming its base model (6.40) and specialized baselines like MangaLMM (6.60).
- Inference Efficiency: MANGO is significantly faster than other active perception models. It averages 5.21 seconds per example on a single NVIDIA A800 GPU, compared to 30.02 seconds for DeepEyes and 70.49 seconds for ViGoRL, making it highly practical for real-world deployment.
- Response Compactness: MANGO concentrates its answers in high-score bins while maintaining a much shorter response token length than its competitors, proving that it avoids generating redundant intermediate text.
Limitations and Future Work
- Resource Constraints: The evaluation is constrained by the limited availability of large-scale manga VQA datasets. Current benchmarks only cover a narrow slice of genres, eras, languages, and cultural contexts.
- Future Directions: The authors suggest that future work should focus on developing broader benchmarks and more comprehensive evaluation protocols to test model behavior across more diverse and culturally nuanced manga scenarios.
Summary updated