Abstract
Recent advancements in face swapping have achieved high fidelity in localized facial features, yet the more complex task of "head swapping"—which encompasses the entire cranial structure, hairstyle, and skin tone—remains a significant challenge. We present Head Injection Diffusion (HID), a novel zero-shot framework designed for seamless head swapping in diverse, real-world scenarios. By leveraging a novel Inverse-Orthogonal Masking (IOMask) technique and a dedicated Hair Injection Module, HID effectively integrates source identity into target bodies without the need for person-specific fine-tuning. Our method demonstrates superior performance in preserving identity and handling complex occlusions and profiles compared to existing baselines.
1. Introduction
While traditional face swapping focuses on the "inner face" (eyes, nose, and mouth), head swapping requires the integration of the entire head—including hair, face shape, and skin tone—onto a new body. This task is significantly more demanding as it requires the synthesis of a consistent cranial structure that matches the target body's pose and lighting while preserving the source's unique identity.
Existing approaches often suffer from two primary issues: dataset bias and masking inaccuracies. Many models are trained on face-centered, frontal-facing images, causing them to fail when applied to real-world photos with diverse angles. Furthermore, traditional masking techniques often create harsh boundaries or fail to account for complex hair strands, leading to unrealistic transitions between the new head and the target body. To address these issues, we introduce Head Injection Diffusion (HID), a zero-shot pipeline that utilizes the generative power of diffusion models to perform high-fidelity head swaps in unconstrained environments.
2. Methodology
The HID framework is built upon the identity-preserving capabilities of PhotoMaker V2 and introduces three primary innovations to ensure seamless blending and identity retention.
2.1 IOMask (Inverse-Orthogonal Mask)
A critical challenge in head swapping is determining the precise region to modify. Simple pixel-wise or latent-wise differences often fail to capture the nuances of hair strands or complex backgrounds. HID introduces IOMask, which utilizes orthogonal projection in the latent noise space. By isolating the vector components of the "head condition" that are perpendicular to the "body condition," the model identifies the semantic region of the head with high precision. This context-aware masking allows for the integration of complex hairstyles and ensures that the boundary between the new head and the original body remains natural, even in side profiles.
2.2 Hair Injection Module
Hairstyle is a fundamental component of human identity. To prevent the model from generating generic hair, HID incorporates a dedicated Hair Injection Module. This module uses a CLIP image encoder and a specialized Hair Encoder to extract fine-grained features from the source image. These features are then fused into the text embeddings, guiding the diffusion process to replicate the source's specific hairstyle, texture, and volume rather than relying on a generic approximation.
2.3 Zero-Shot Pipeline
The HID pipeline is entirely zero-shot, requiring no additional training or fine-tuning for new subjects. The process begins with a source head and a target body. DDIM inversion is employed to map the target image into the latent space. During the denoising process, the IOMask ensures that only the head region is updated with the source identity features provided by the Hair Injection Module and the identity-preserving backbone.
3. Experiments and Results
3.1 Dataset and Training
The model was trained using the SHHQ-1.0 dataset, which contains 40,000 high-quality human images showing the full upper body. Images were preprocessed to 1024x1024 resolution using GFPGAN for high-resolution training. To facilitate training, the authors utilized LLaVa-NEXT to generate descriptive captions, which were then modified to include trigger words for the identity and hair injection modules.
3.2 Quantitative and Qualitative Evaluation
HID was compared against REFace, a leading zero-shot head swapping baseline. HID demonstrated superior performance across multiple metrics:
- FID (Fréchet Inception Distance): HID achieved a score of 37.18, significantly lower than REFace's 40.71, indicating higher overall image quality and realism.
- Identity Preservation: HID outperformed baselines in CLIP-I and LPIPS metrics for both the head and hair regions.
Qualitatively, visual results show that HID handles side views and long hair much more gracefully than previous methods. While earlier techniques often left "ghosting" artifacts of the original hair or created unnatural skin transitions, HID achieves a "seamless melt" that respects the target body's pose (via ControlNet) and skin tone.
4. Limitations and Future Work
Despite its success, HID has limitations. The masking process can occasionally affect fine details in peripheral regions, such as the hands or neck, if they are included in the mask. For example, a specific hand pose might be slightly altered during the blending process. Additionally, while the model performs well on upper-body swaps, extending the framework to full-body scenarios—including shoes, gait, and complex backgrounds—is the next logical step. Finally, as a diffusion-based method involving DDIM inversion, HID is computationally more intensive than simpler GAN-based face swappers.
5. Conclusion
HID represents a significant step forward in the field of digital identity manipulation. By treating the head as a holistic unit and introducing novel latent-space masking techniques, HID achieves a level of realism in head swapping that was previously unattainable in zero-shot, real-world scenarios.
Summary updated