Vectorization of Raster Manga by Deep Reinforcement Learning

Hao Su, Jianwei Niu, Xuefeng Liu, Jiahe Cui, Ji WanarXiv 2021 · arXiv

Introduction

Manga, a cornerstone of global visual culture, is characterized by its unique artistic style involving intricate line work, varying stroke weights, and complex shading patterns known as screentones. While vector graphics offer significant advantages for digital distribution—such as infinite scalability and high compression—converting existing raster-based manga into high-quality vectors remains a significant technical challenge. Traditional algorithmic approaches often fail to capture the nuance of professional manga, while standard deep learning models struggle with the high-parameter complexity of full-page illustrations.

Comparison of MARVEL's vectorization results across various popular manga styles including One Piece, Naruto, and Bleach.

The MARVEL Framework

The MARVEL (MAnga’s Raster to VEctor Learning) framework addresses these challenges by reframing vectorization as a sequential decision-making process. Instead of attempting a one-step translation from pixels to vectors, MARVEL employs a Deep Reinforcement Learning (DRL) agent that "draws" the image stroke-by-stroke.

Sequential Stroke Generation

Utilizing an Actor-Critic architecture (specifically DDPG), the agent predicts parameters for Quadratic Bezier Curves (QBC). Each predicted action defines the coordinates, thickness, and color of an individual stroke. This iterative approach allows the model to refine the canvas progressively, correcting previous inaccuracies and building up complex textures that single-step models often miss.

Optimization and Pruning

To ensure high fidelity and efficiency, MARVEL incorporates several key mechanisms:

  • Reward Function: A multi-part reward system encourages the agent to maximize visual similarity to the ground truth while ensuring strokes are meaningful and efficient.
  • Patch-based Processing: High-resolution pages are divided into patches, processed individually, and then seamlessly synthesized.
  • Pruning Module: To prevent the accumulation of redundant data, a pruning module iteratively removes strokes that do not significantly contribute to image quality, reducing final file sizes by over 50%.

Experimental Results

The framework was evaluated using the DeepManga dataset, comprising over 42,000 images. MARVEL achieved state-of-the-art performance, outperforming existing methods like Im2Vec and LIVE in both quantitative metrics and qualitative user studies.

Detailed input vs. output comparison for complex Jojo's Bizarre Adventure artwork, highlighting the preservation of fine lines and shading.

Quantitatively, MARVEL reached a Mean Square Error (MSE) of 0.0825 and a Structural Similarity Index (SSIM) of 0.8968. Qualitatively, user studies indicated a strong preference for MARVEL’s outputs over commercial tools like Adobe Illustrator, citing superior visual pleasantness and accuracy. Furthermore, MARVEL demonstrates significant computational efficiency, vectorizing complex pages in under three minutes—a task that takes competing stepwise methods hours to complete.

Conclusion

MARVEL represents a significant advancement in manga vectorization, successfully bridging the gap between raster complexity and vector efficiency. By modeling the process as a sequential drawing task and incorporating robust pruning, it delivers high-fidelity results suitable for professional use. Future work will focus on extracting centerlines for animation and improving the procedural recognition of screentones.

Summary updated

Abstract

Manga is a fashionable Japanese-style comic form that is composed of black-and-white strokes and is generally displayed as raster images on digital devices. Typical mangas have simple textures, wide lines, and few color gradients, which are vectorizable natures to enjoy the merits of vector graphics, e.g., adaptive resolutions and small file sizes. In this paper, we propose MARVEL (MAnga's Raster to VEctor Learning), a primitive-wise approach for vectorizing raster mangas by Deep Reinforcement Learning (DRL). Unlike previous learning-based methods which predict vector parameters for an entire image, MARVEL introduces a new perspective that regards an entire manga as a collection of basic primitives\textemdash stroke lines, and designs a DRL model to decompose the target image into a primitive sequence for achieving accurate vectorization. To improve vectorization accuracies and decrease file sizes, we further propose a stroke accuracy reward to predict accurate stroke lines, and a pruning mechanism to avoid generating erroneous and repeated strokes. Extensive subjective and objective experiments show that our MARVEL can generate impressive results and reaches the state-of-the-art level. Our code is open-source at: https://github.com/SwordHolderSH/Mang2Vec.
BibTeX
@misc{su2021vectorization,
  author = {Hao Su and Jianwei Niu and Xuefeng Liu and Jiahe Cui and Ji Wan},
  title = {{Vectorization of Raster Manga by Deep Reinforcement Learning}},
  year = {2021},
  eprint = {2110.04830},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.04830},
}
Figure