Introduction
The proliferation of "cartoon filters" has made facial stylization a common feature in social media. However, extending this capability to full-body human images remains a significant technical hurdle. Unlike facial images, which are relatively constrained in geometry, full-body images involve a vast array of poses, complex clothing textures, and diverse backgrounds. This paper presents a novel framework for one-shot full-body stylization, enabling the transformation of a real-world photo into the aesthetic of a single artistic reference image while preserving the subject's pose and identity.
The Challenge of Full-Body Stylization
The authors identify two primary gaps in existing style transfer methods when applied to the human form:
- Structural Complexity and Entanglement: Full-body images feature intricate details like fabric folds and limb positions. Standard generative models often "tangle" pose and appearance, meaning that changing the style (e.g., to a sketch or anime) often results in a loss of the original pose or clothing structure.
- Non-Human Body Proportions: Artistic styles frequently rely on exaggerated geometry. For instance, characters in films like Frozen or Coraline possess head-to-body ratios and limb lengths that differ drastically from realistic human anatomy. Traditional style transfer focuses on color and texture (the "look"), leaving the realistic skeleton intact and creating an "uncanny valley" effect where the character looks like a human wearing a mask rather than a cohesive artistic creation.
Proposed Methodology
To address these challenges, the researchers developed a three-stage pipeline centered on a pose-guided generator.
The "Destylization" Trick
A major hurdle in training stylization models is the lack of paired data (i.e., a photo of a person and a corresponding drawing of that same person in the same pose). To circumvent this, the authors propose a "destylization" step. They use a pre-trained human generator ($G_0$) to project the artistic reference image into a photo-realistic space. This creates a synthetic "photo" version of the artistic character, providing a pseudo-paired training example.
Joint Fine-tuning and Warping
With the synthetic pair in hand, a generator ($G_a$) is fine-tuned to map the destylized photo back to the original artistic style. To prevent the model from overfitting to the single reference image, the authors employ a "joint task" strategy: the model must simultaneously learn to stylize a diverse dataset of real human photos using standard perceptual and style losses.
The base architecture is an enhanced version of the "Dressing in Order" (DiOr) model, dubbed DiOr+. The authors found that warping-based generators, which physically shift pixels, are superior to encoder-decoder models for preserving complex clothing patterns during this process.
Learnable Skeleton Deformation
Perhaps the most significant innovation is the skeleton deformation module. The system uses a Multi-Layer Perceptron (MLP) to predict "body ratio factors" based on the artistic reference. Before the final image generation, the input person's skeleton is rescaled to match the proportions of the target style. This allows the model to generate characters with the large heads or elongated limbs characteristic of specific art styles.
Results and Evaluation
The method was tested against a gallery of 26 artistic references, including anime (K-On!), films (Frozen), and classical art (Modigliani).
Qualitative and Quantitative Success
The system successfully handles "one-shot" cases, maintaining the subject's specific clothing and pose while convincingly applying the character's aesthetic and proportions. In a user preference study, the proposed method was preferred over NNST (Neural Neighbor Style Transfer) 76.1% of the time. Quantitatively, the model achieved significantly lower LPIPS scores on art reconstruction tasks compared to other state-of-the-art models like ADGAN or NTED, indicating a higher fidelity to the target style.
Conclusion and Limitations
While the framework represents a significant step forward, it is not without limitations. The output quality is inherently tied to the capabilities of the base generator; if the base model struggles with rendering hands or overlapping limbs, those artifacts will persist. Additionally, the "destylization" step can struggle with highly abstract or grayscale art, occasionally failing to learn how to "decolorization" a photo. Future work may focus on integrating more robust base models and improving the handling of non-color-based artistic elements.
Summary updated