Layout Analysis of Tree-Structured Scene Frames in Comic Images

Takamasa Tanaka, Kenji Shoji, Fubito Toyama, Juichi MiyamichiIJCAI 2007

Abstract

This paper presents an automated method for segmenting comic book pages into individual scene frames by modeling the layout as a tree structure created through recursive "guillotine cuts." By using density gradients and a preprocessing step that masks frame interiors, the researchers achieved an 80% success rate in correctly identifying frames and their reading order, even when drawings or speech balloons overlap frame boundaries.

Introduction

As digital comic consumption shifted toward mobile devices with small screens, a significant challenge emerged: converting traditional print layouts into a readable mobile format. Simply shrinking a full page often makes text unreadable. To solve this, software must "understand" the page layout—specifically, where each frame starts and ends, and the correct sequence in which they should be read. Standard document layout analysis often fails because comic frames are frequently non-rectangular and artistic elements often "break" the frame borders, confusing traditional edge-detection algorithms.

Methodology

The authors propose a multi-step approach based on the concept of recursive guillotine cuts. This assumes a comic page can be divided by straight lines into regions, which can be further subdivided, forming a hierarchical tree structure.

Preprocessing and Line Detection

To prevent drawings and speech balloons from interfering with line detection, the authors first approximate the outlines of the artwork and fill these quadrangular regions with black. This leaves the "gutters" (the white space between frames) as the primary features. They then calculate the density gradient of the image to find straight lines where the gradient accumulation is highest. A Gaussian function is used to give more weight to the central part of a region, as frame boundaries are less likely to be at the very edges of a previously defined segment.

Tree Construction

Each successful cut adds a node to a tree. Divisions are categorized as Horizontal (H) or Vertical (V). The system also includes specific logic to handle "T-junctions," where a line might split only a portion of a region rather than cutting all the way through the page.

The tree structure represents the hierarchical division of the page into frames using horizontal (H) and vertical (V) cuts.

Experimental Results

The study evaluated the algorithm using a dataset of 672 pages from four different print comic booklets, including titles like Ranma 1/2, Dragon Ball, and Slam Dunk. The proposed method successfully divided 79.5% (534 out of 672) of the test pages. This was a significant improvement from previous methods (63.4%), with the preprocessing step alone accounting for a 16% improvement in accuracy.

Key Insights

Modeling a complex comic page as a series of recursive splits simplifies a difficult computer vision problem into a structured tree-search problem. By linking the layout analysis directly to a tree structure, the reading order is solved automatically as a byproduct of the segmentation; a depth-first search on the tree naturally follows the standard flow of Japanese manga.

A comparison between a segmented comic page and its corresponding logical tree structure, illustrating how physical layout maps to data.

Conclusion and Future Work

While the system is robust for slanted frames, it still struggles with complex overlaps that break the "guillotine cut" logic (e.g., a character's limb crossing multiple frames) or nested frames. Future research aims to refine the detection of single-frame pages and handle flashback sequences, which often use black gutters instead of white ones.

Summary updated

BibTeX
@inproceedings{tanaka2007layout,
  author = {Takamasa Tanaka and Kenji Shoji and Fubito Toyama and Juichi Miyamichi},
  title = {{Layout Analysis of Tree-Structured Scene Frames in Comic Images}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2007},
}
Figure