Introduction
Hair transfer—the process of migrating the color, shape, and structure of hair from a reference image to a target subject—remains a significant challenge in computer vision. While previous iterations using Generative Adversarial Networks (GANs) and single-view diffusion models have shown promise, they often struggle with temporal and spatial consistency. When the camera angle changes, these models frequently exhibit "flickering" or identity warping. Stable-Hair v2 addresses these issues by introducing a diffusion-based framework capable of maintaining perfect multi-view consistency for 3D-aware virtual try-ons.
The primary obstacles in this field are twofold: the scarcity of high-quality "triplet" datasets (same person, different hairstyles, multiple synchronized views) and the technical difficulty of ensuring view consistency across rotations. Stable-Hair v2 overcomes these through a novel synthetic data generation pipeline and a multi-stage training strategy.
Methodology
The proposed framework is divided into a sophisticated data generation pipeline and a three-stage model architecture designed to decouple pose, identity, and style.
Synthetic Data Generation
To address data scarcity, the authors developed an automated pipeline to create synthetic training triplets. This involves:
- Diffusion-based Bald Converter: A module that removes hair from source images to create a "clean slate" while preserving the subject's facial identity.
- Data-Augmented Inpainting: This stage generates diverse reference images. By utilizing a "face-and-hair mask" and swapping backgrounds, the model is forced to focus on hair features rather than environmental cues.
- Face-Finetuned Multi-View Diffusion: The researchers fine-tuned a general-purpose 3D diffusion model (SV3D) on 20,000 facial videos to ensure high-quality, consistent head rotations.
Model Architecture and Training
The transfer model employs a three-stage training approach to ensure stability and fidelity.
- Stage 1: Pose-Controllable Latent IdentityNet: This module learns to control the viewpoint (azimuth and polar angles) while maintaining facial consistency. Training on bald images first allows the model to learn the underlying head geometry without the interference of complex hair structures.
- Stage 2: Hair Extractor: A dedicated module extracts fine-grained details from the reference hairstyle and injects them into the main diffusion model via cross-attention layers.
- Stage 3: Temporal Attention: Drawing from video generation techniques, temporal layers are added to ensure smooth, flicker-free transitions between different viewpoints.
Key Insights
A critical finding was the necessity of decoupling pose and style. Simultaneous training often led to structural artifacts, such as "melted" head shapes. By isolating pose alignment on bald subjects, the model establishes a stable geometric foundation. Additionally, the use of a Latent IdentityNet—operating in the VAE's compressed space—proved superior to pixel-space ControlNets for maintaining color consistency during rotation. Finally, the "anti-shortcut" data augmentation strategy prevents the model from relying on background or clothing colors to identify hair regions.
Evaluation and Results
Stable-Hair v2 was evaluated using the FFHQ dataset and a custom synthetic multi-view dataset comprising 20,000 video sets. Performance was measured against state-of-the-art methods like Barbershop and HairFastGAN.
The model achieved superior results across all key metrics:
- Fidelity: Lowest FID (35.125), indicating top-tier visual quality.
- Style Accuracy: Highest CLIP-I (0.431), showing the closest match to reference hairstyles.
- Identity Preservation: Highest IDS (0.778), ensuring the target subject remains recognizable.
User studies further confirmed these findings, with participants consistently rating Stable-Hair v2 highest for naturalness, accuracy, and identity preservation.
Conclusion
Stable-Hair v2 represents a significant step forward in 3D-aware virtual hair try-ons. By combining synthetic data generation with a multi-stage diffusion approach, it achieves unprecedented multi-view consistency. Future work will focus on expanding the view range to full 360-degree rotations and improving background reconstruction in complex scenes. While computationally more intensive than GAN-based alternatives, the leap in realism and stability makes it a robust solution for high-fidelity digital makeovers.
Summary updated