Abstract
Recent years have witnessed increasing attention in cartoon media, powered by the strong demands of industrial applications. As the first step to understand this media, cartoon face recognition is a crucial but less-explored task with few datasets proposed. In this work, we first present a new challenging benchmark dataset, consisting of 389,678 images of 5,013 cartoon characters annotated with identity, bounding box, pose, and other auxiliary attributes. The dataset, named iCartoonFace, is currently the largest-scale, high-quality, rich-annotated, and spanning multiple occurrences in the field of image recognition, including near-duplications, occlusions, and appearance changes. In addition, we provide two types of annotations for cartoon media, i.e., face recognition, and face detection, with the help of a semi-automatic labeling algorithm. To further investigate this challenging dataset, we propose a multi-task domain adaptation approach that jointly utilizes the human and cartoon domain knowledge with three discriminative regularizations. We hence perform a benchmark analysis of the proposed dataset and verify the superiority of the proposed approach in the cartoon face recognition task.
Introduction
While human face recognition has reached near-perfect accuracy (exceeding 99% on benchmarks like LFW), recognizing cartoon characters remains a significant challenge. This performance gap stems from two primary factors: data scarcity and domain complexity. Existing cartoon datasets are often small, noisy, or restricted to specific styles like Japanese manga. Furthermore, cartoon faces exhibit extreme variations in artistic style, "near-duplication" (where distinct characters appear nearly identical), and exaggerated expressions that defy standard facial geometry.
To address these issues, we introduce iCartoonFace, a massive, high-quality dataset designed to push the boundaries of stylized character recognition. Our approach is rooted in the concept of anthropomorphism: since artists abstract real human features to create cartoon faces, we can leverage vast knowledge of human facial structures to help machines understand their stylized counterparts.
The iCartoonFace Dataset
iCartoonFace consists of 389,678 images across 5,013 identities. To construct this dataset efficiently, we employed a three-stage semi-automatic assembly pipeline:
- Hierarchical Collection: Images were scraped based on album and character names.
- Guided Data Filtering: Pre-trained human face recognition models served as a "coarse filter" to eliminate non-face images and duplicates.
- Q/A Manual Annotation: A human-in-the-loop system allowed annotators to quickly verify identity matches, significantly accelerating the labeling process.
The resulting dataset includes rich annotations for bounding boxes, 3D poses (yaw, pitch, roll), and gender, making it orders of magnitude larger and more diverse than existing benchmarks like Manga109 or WebCaricature.
Multi-task Domain Adaptation (MTDA)
To bridge the gap between human and cartoon domains, we propose a Siamese-style network that trains on human and cartoon data simultaneously. The framework utilizes three specific loss functions:
- Classification Loss: Standard supervised learning for both domains.
- Unknown Identity Rejection (UIR) Loss: An unsupervised regularization technique that improves model generalization to unseen identities.
- Domain Adaptation (DA) Loss: An adversarial loss designed to align high-level features between human and cartoon faces, effectively transferring robust structural knowledge from the human domain to the cartoon domain.
Key Insights and Results
Our analysis revealed several critical aspects of cartoon recognition:
- The "Context" Boost: Unlike human face recognition, cartoon recognition benefits significantly from surrounding context like hair, hats, and accessories. Expanding the crop area around the face improved Rank@1 accuracy from 71.15% to 78.97%, suggesting that a character's silhouette is often more unique than the face itself.
- Near-Duplication Challenge: Many characters share a "same-face" design style, forcing models to focus on subtle local differences that standard human models might overlook.
In benchmarking, our F-ArcFace (MTDA) approach achieved a 84.34% Rank@1 accuracy, outperforming standard ArcFace (83.34%) and baseline SoftMax (78.97%). While this represents a strong start, the 15% gap compared to human face performance highlights the ongoing difficulty of the cartoon domain.
Conclusion
iCartoonFace provides a robust foundation for future research in stylized image recognition. By combining a massive, high-quality dataset with domain adaptation techniques, we demonstrate that human facial knowledge is a powerful tool for understanding cartoon media. Future work will focus on narrowing the "style gap" for more abstract or non-humanoid characters and improving performance under heavy occlusion.
Summary updated