ManGo: Manga Active Narrative Grounding Optimization

Hao Qiu, Junyan Wang, Zheyuan Liu, Lei Fan, Hong Jia, Lianbo Guo, Zhulin TaoarXiv 2026 · arXiv

TL;DR

MANGO (Manga Active Narrative Grounding Optimization) is an unsupervised reinforcement learning framework that transforms Manga Visual Question Answering from a passive full-page reading task into an active, question-guided panel exploration. By iteratively building a compact "evidence sketch" and optimizing it via group-relative policy training with self-ranking and path-consistency rewards, MANGO achieves state-of-the-art performance without requiring human-annotated answers or rationale paths.

What Problem Does This Paper Address?

Reading a manga page isn't like looking at a standard natural image; it is a highly structured multimodal narrative. Information is non-linearly distributed across ordered panels, embedded dialogue bubbles, recurring characters, and implicit transitions between events.

Standard Multimodal Large Language Models (MLLMs) typically process an entire manga page in a single passive pass. This often causes them to miss fine-grained details, hallucinate connections, or generate overly verbose, redundant answers. While "active perception" (allowing a model to dynamically zoom or crop into regions) can help, training models to navigate these complex layouts usually requires expensive, human-annotated step-by-step reasoning paths (rationales), which are incredibly difficult to scale.

Methods

The authors introduce MANGO, a framework that reframes manga VQA as an iterative, question-driven evidence-seeking process. It couples two core components:

  1. Active Narrative Sketching (ANS): Instead of reading panels in a fixed sequence, the model dynamically decides which panel to inspect next based on the question and its current knowledge state. At each step, it performs three actions: selects a panel ID, extracts a concise, grounded text clue, and decides whether to continue exploring or stop and answer. This builds a compact "evidence sketch."
  2. Unsupervised Reward Modeling & GRPO: To train this behavior without gold-standard human answers, MANGO uses Group-Relative Policy Optimization (GRPO)—a critic-free reinforcement learning variant—driven by two unsupervised rewards:
  3. Answer Preference Reward ($R_{pref}$): The model samples multiple reasoning paths (rollouts), clusters identical final answers, and uses the policy model itself to rank the unique candidates (listwise self-ranking).
  4. Path Consistency Reward ($R_{cons}$): It organizes the panel trajectories of high-ranking rollouts into a prefix tree, rewarding paths that consistently follow stable, reliable sequences of crucial panels.
Figure 2: Overview of MANGO. The upper part shows Active Narrative Sketching, where the model performs panel selection, evidence sketching, and stopping decisions to build a compact sketch S before answering. The lower part shows unsupervised training: sampled rollouts are scored by answer preference and path consistency rewards, which are combined for group-relative policy optimization. ©Mikio Yoshimori

Datasets

  • Training: A question-only subset of 512 questions from MangaVQA, utilizing only the questions and Manga109 panel bounding box annotations. Crucially, no ground-truth answers or human rationales were used to compute training rewards.
  • Evaluation: Evaluated across three major benchmarks:
  • MangaVQA: Tests open-ended exact extraction and descriptive answering.
  • ChrOMIC: Evaluates chronological reasoning across multi-panel comics (Japanese manga subset) in both multiple-choice and open-ended formats.
  • MangaUB: A large-scale benchmark probing single-panel evidence extraction and multi-panel evidence aggregation.

Key Insights and Ideas

  • Question-Directed Order vs. Reading Order: The optimal path to answer a specific question often deviates from the standard panel reading order (e.g., jumping ahead to see an outcome, then back to identify a speaker). ANS allows the model to dynamically restructure the narrative flow based on informational need.
  • Conciseness Over Verbosity: Unlike other active perception methods that accumulate lengthy visual descriptions, MANGO forces the model to extract atomic, local clues. This structured brevity prevents the model from getting lost in redundant data, leading to higher final accuracy.
  • Self-Supervised Rationales via Consistency: By rewarding trajectories that form stable prefixes in a tree structure, the framework successfully discovers true causal paths across panels entirely through self-supervision.

Key Results

  • State-of-the-Art Performance: Built on a Qwen3-VL-8B backbone, MANGO consistently outperformed both general perception MLLMs and existing active perception baselines. On MangaVQA, it achieved an overall score of 6.90, outperforming its base model (6.40) and specialized baselines like MangaLMM (6.60).
  • Inference Efficiency: MANGO is significantly faster than other active perception models. It averages 5.21 seconds per example on a single NVIDIA A800 GPU, compared to 30.02 seconds for DeepEyes and 70.49 seconds for ViGoRL, making it highly practical for real-world deployment.
  • Response Compactness: MANGO concentrates its answers in high-score bins while maintaining a much shorter response token length than its competitors, proving that it avoids generating redundant intermediate text.

Limitations and Future Work

  • Resource Constraints: The evaluation is constrained by the limited availability of large-scale manga VQA datasets. Current benchmarks only cover a narrow slice of genres, eras, languages, and cultural contexts.
  • Future Directions: The authors suggest that future work should focus on developing broader benchmarks and more comprehensive evaluation protocols to test model behavior across more diverse and culturally nuanced manga scenarios.

Summary updated

Abstract

Manga visual question answering requires models to answer questions over panel-based visual narratives, where relevant evidence is distributed across ordered panels, embedded text, recurring characters, and implicit event transitions. This structure makes passive page encoding insufficient, as the model must identify which panels to inspect, what clues to retain, and when the accumulated evidence is sufficient for answering. We propose ManGo (Manga Active Narrative Grounding Optimization), an unsupervised framework for active manga visual question answering. ManGo introduces Active Narrative Sketching (ANS), which iteratively selects panels, extracts concise grounded clues, and decides when to stop, forming a compact question-directed evidence sketch before answer generation. To optimize this behavior without human-annotated answers or rationale paths, ManGo samples multiple ANS rollouts and applies group-relative training with two rewards: answer preference from listwise self-ranking and path consistency from stable ordered panel trajectories. The combined reward is optimized with group-relative policy training, encouraging the model to improve both final answers and the panel-level evidence paths that support them. Experiments on standard manga understanding benchmarks show that ManGo achieves state-of-the-art performance across different settings.
BibTeX
@misc{qiu2026mango,
  author = {Hao Qiu and Junyan Wang and Zheyuan Liu and Lei Fan and Hong Jia and Lianbo Guo and Zhulin Tao},
  title = {{ManGo: Manga Active Narrative Grounding Optimization}},
  year = {2026},
  eprint = {2608.29865},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.29865},
}
Figure