Adversarial Colorization Of Icons Based On Structure And Color Conditions

Tsai-Ho Sun, Chien-Hsun Lai, Sai-Keung Wong, Yu-Shuen Wang

Abstract

This paper presents a dual conditional Generative Adversarial Network (GAN) designed for the colorization of hand-drawn icon contours. By decoupling the discriminator's objectives into structural fidelity and color style consistency, the proposed model overcomes the challenges of domain diversity inherent in icon design. Experimental results demonstrate that this approach produces professional-quality, leak-proof icons that significantly outperform existing domain-transfer and manga-colorization models in both qualitative and quantitative assessments.

Introduction

Designing effective icons requires a delicate balance of shape, structure, and color consistency. While machine learning has seen success in colorizing black-and-white Manga and Anime line art, these models often fail when applied to icons due to extreme domain diversity. Unlike Manga characters, which share common structural features like faces and eyes, icons can represent any object, from a magnifying glass to a scooter.

Standard GAN discriminators struggle to learn a universal distribution for such diverse structures, leading to artifacts, color bleeding, or outputs that ignore input sketches. This paper addresses these issues by introducing a dual conditional GAN architecture that treats structure and color as independent tasks.

Methodology

To resolve discriminator confusion, the authors propose a dual conditional GAN. Instead of a single discriminator, the system employs two targeted adversarial networks:

  1. Structure Discriminator ($D_s$): Focuses exclusively on shape. It evaluates whether the generated icon's structure matches the input contour.
  2. Color Discriminator ($D_c$): Focuses on palette and style. It assesses whether the generated icon aligns with the color style of a provided reference.
The architecture of the dual conditional GAN, showing the generator (G) and the split structure ($D_s$) and color ($D_c$) discriminators.

Training and Style Mapping

To prevent the network from simply memorizing images, the training data was pre-processed using K-means clustering (K=500) on 3D Lab color histograms. The color discriminator is trained using pairs of different icons from the same cluster, teaching it to recognize stylistic patterns rather than exact pixel layouts. For the user interface, these clusters are mapped to semantic labels like "Casual," "Clear," or "Gorgeous," allowing users to select a style intuitively.

Datasets and Implementation

The researchers compiled a custom dataset of 12,575 multi-color icons from flaticon.com. The training pipeline is fully self-supervised: structural contours were extracted using Canny edge detection, and color styles were derived from 3D Lab color histograms. This automated approach eliminates the need for manual labeling.

Key Results and Discussion

The proposed method generates clean, flat-colored icons that respect boundaries and resemble human-designed assets. Unlike traditional bucket-fill tools, the network is inherently "leak-proof," inferring intended boundaries even from open contours based on learned icon distributions.

A grid demonstration showing the model's ability to apply various color styles (rows) to a wide range of structural contours (columns).

Quantitative and Qualitative Evaluation

In quantitative tests, the model achieved superior scores in structural distance (edge matching) and color distance (Jensen-Shannon divergence) compared to baselines like CycleGAN, MUNIT, and iGAN. A blind user study with 92 participants further validated these results, with the proposed method receiving a top score of 3.65 out of 5, significantly higher than the next best method, Comicolorization (3.27).

Limitations and Future Work

Despite its success, the model is currently "semantics-blind," meaning it does not understand the object it is colorizing (e.g., it might color an apple blue if a "cool" style is selected). It also occasionally struggles with negative space, such as the holes in a bicycle frame. Future work will explore incorporating semantic object labels to resolve conflicts between structure and color conditions, ensuring more contextually appropriate colorizations.

Summary updated

Abstract

We present a system to help designers create icons that are widely used in banners, signboards, billboards, homepages, and mobile apps. Designers are tasked with drawing contours, whereas our system colorizes contours in different styles. This goal is achieved by training a dual conditional generative adversarial network (GAN) on our collected icon dataset. One condition requires the generated image and the drawn contour to possess a similar contour, while the other anticipates the image and the referenced icon to be similar in color style. Accordingly, the generator takes a contour image and a man-made icon image to colorize the contour, and then the discriminators determine whether the result fulfills the two conditions. The trained network is able to colorize icons demanded by designers and greatly reduces their workload. For the evaluation, we compared our dual conditional GAN to several state-of-the-art techniques. Experiment results demonstrate that our network is over the previous networks. Finally, we will provide the source code, icon dataset, and trained network for public use.
BibTeX
@inproceedings{sun2019adversarial,
  author = {Tsai-Ho Sun and Chien-Hsun Lai and Sai-Keung Wong and Yu-Shuen Wang},
  title = {{Adversarial Colorization Of Icons Based On Structure And Color Conditions}},
  booktitle = {ACM Multimedia},
  year = {2019},
  doi = {10.1145/3343031.3351041},
}
Figure