Introduction
Identity swapping, or face swapping, aims to generate a face that inherits the identity of a source image while preserving the perceptual attributes—such as pose, lighting, and expression—of a target image. A persistent challenge in this field is entanglement, where the target's facial geometry (e.g., jawline or eye shape) "leaks" into the final result, creating an ambiguous hybrid identity.
Existing subject-agnostic methods often rely on implicit guidance through multiple loss functions, which frequently fails to achieve clean separation. This paper introduces InfoSwap, a framework that explicitly models disentanglement using the Information Bottleneck (IB) principle to extract "minimal sufficient statistics" for identity, thereby minimizing leakage from the target.
Methodology: The InfoSwap Framework
The InfoSwap architecture is built around a pre-trained face recognition model (ArcFace) and consists of two primary modules designed to enforce explicit disentanglement.
Informative Identity Bottleneck (IIB)
The IIB module treats disentanglement as an information compression task. It employs "information controllers" ($\lambda$) that dynamically add noise to the internal features of the pre-trained backbone. By optimizing for the minimal amount of data required to represent identity, the model learns to discard identity-irrelevant information (like pose and expression) as noise.
Adaptive Information Integrator (AII)
Once the identity is compressed, the AII module integrates this source identity with the target's perceptual features. Utilizing a U-Net decoder and Adaptive Instance Normalization (AdaIN), the AII synthesizes a high-resolution image that blends these disparate components seamlessly.
Identity Contrastive Loss (ICL)
To further refine the identity, the authors introduce the Identity Contrastive Loss. This loss function ensures the generated face is close to the source identity in the latent space while remaining distinctly far from the target identity. This prevents the model from settling for a "middle-ground" face that lies on the angle bisector between the two identities.
Experimental Results
The model was trained on the FFHQ and CelebA-HQ datasets (96,000 images at 512x512 resolution) and evaluated on the FaceForensics++ (FF++) benchmark.
Qualitative Analysis
InfoSwap demonstrates superior preservation of source-specific traits compared to state-of-the-art (SOTA) methods. As shown in the comparison below, the model effectively captures nuances such as face shape, cheekbone structure, and chin length, which are often lost or blended in competing frameworks like FaceShifter or FSGAN.
Quantitative Performance
- ID Retrieval: InfoSwap achieved a record ID retrieval rate of 99.67% on FF++, significantly outperforming FaceShifter (97.38%) and FSGAN (60.41%).
- Identity Discriminativeness: Statistical analysis revealed that InfoSwap is the only method where the identity similarity distribution of generated faces has zero overlap with the target identity distribution.
- User Preference: In a study of 50 users, InfoSwap was rated as "most identical to source" 51.1% of the time, more than double the score of the next best competitor.
Discussion and Future Work
While InfoSwap excels at identity preservation, the authors note that methods like FSGAN may occasionally show lower expression errors by only swapping the inner face. However, InfoSwap’s whole-face approach provides a more realistic and consistent result. Beyond identity swapping, the high-fidelity outputs of InfoSwap could serve as a valuable resource for training more robust forgery detection systems. The researchers acknowledge the potential for misuse and emphasize that their work on discriminative identities can contribute to developing defensive technologies against malicious deepfakes.
Summary updated