Abstract
Due to unreliable geometric matching and content misalignment, most conventional pose transfer algorithms fail to generate fine-trained person images. In this paper, we propose a novel framework Spatial Content Alignment GAN (SCA-GAN) which aims to enhance the content consistency of garment textures and the details of human characteristics. We first alleviate the spatial misalignment by transferring the edge content to the target pose in advance. Secondly, we introduce a new Content-Style DeBlk which can progressively synthesize photo-realistic person images based on the appearance features of the source image, the target pose heatmap and the prior transferred content in edge domain. We compare the proposed framework with several state-of-the-art methods to show its superiority in quantitative and qualitative analysis. Moreover, detailed ablation study results demonstrate the efficacy of our contributions.
Introduction
Pose transfer—the task of synthesizing an image of a person in a new pose based on a source image—is notoriously difficult because the source and target images are "spatially misaligned." Existing methods generally fall into two camps: warping-based methods that attempt to geometrically stretch the source image, and attention-based methods that use neural networks to attend to relevant source features. However, both often fail to maintain fine-grained details like garment patterns and facial identity due to a lack of reliable spatial guidance.
Methodology
The proposed Spatial Content Alignment GAN (SCA-GAN) addresses these challenges through a two-phase framework that prioritizes structural alignment before final synthesis.
Phase 1: Prior Content Transfer Network (PCT-Net)
Instead of synthesizing pixels directly, the model first predicts the person's edges in the target pose. Edges represent high-frequency signals that capture the "skeleton" of both the person's identity and their clothing. The PCT-Net takes the source edge map, source pose, and target pose to generate a spatially aligned "blueprint" for the next phase.
Phase 2: Image Synthesis Network (IS-Net)
This network takes the source image and the predicted edges from Phase 1 to create the final photo. It features two novel components:
- Style Encoder Blocks (Style EnBlk): These blocks omit standard normalization layers to preserve "unique pattern statistics," such as specific logos or intricate textures that might otherwise be smoothed out by normalization.
- Content-Style SPADE (CS-SPADE): An evolution of the Spatially-Adaptive Denormalization technique, CS-SPADE simultaneously injects aligned content (target pose and predicted edges) and misaligned style (source image features) during the upsampling process.
Experiments and Results
The researchers evaluated SCA-GAN on the DeepFashion dataset. Quantitatively, SCA-GAN outperformed several state-of-the-art models (including PATN, ADGAN, and APS), achieving the highest Inception Score (IS) of 3.497 and the lowest FID (11.676), indicating superior image quality and perceptual similarity to real photos.
Qualitatively, visual results show much sharper clothing boundaries, realistic creases in garments, and better preservation of facial features compared to competitors, which often produced "blobby" or distorted results in those areas. Ablation studies confirmed that the edge-prediction step is crucial; removing the prior edge transfer caused the FID score to jump from 11.6 to 12.4.
Conclusion
SCA-GAN effectively solves the spatial misalignment problem in pose transfer by utilizing edge maps as an intermediate spatial guide. By decoupling structural content from appearance style, the framework achieves high-fidelity synthesis of person images with consistent textures and sharp details. Codes are publicly available at github.com/rocketappslab/SCA-GAN.
Summary updated