Introduction
Facial attribute editing—tasks such as altering age, gender, or adding accessories like eyeglasses—has traditionally relied on encoder-decoder architectures. Most existing frameworks treat these attributes as local labels, often concatenating a one-hot attribute vector directly onto feature maps. However, this approach faces two significant hurdles: the neglect of global context and the "detail dilemma."
The authors of SSCGAN argue that attributes are not merely local patches but influence the global "style" and statistical distribution of an image. Furthermore, standard skip connections, while useful for preserving identity, often "leak" source attributes into the output, hindering effective editing. Conversely, removing these connections leads to a loss of fine spatial details. SSCGAN addresses these issues by decoupling style manipulation from spatial detail preservation.
Methodology: The SSCGAN Architecture
The core innovation of this work is the Style Skip Connection GAN (SSCGAN). It utilizes a symmetric 6-stage encoder-decoder structure enhanced by two novel modules: Style Skip Connections (SSC) and Spatial Information Transfer (SIT).
Style Skip Connections (SSC)
Instead of transferring raw feature maps, the SSC module extracts channel-wise mean and variance—mathematical representations of "style"—from the encoder. The model calculates a "style residual" based on the difference between source and target attributes. This residual is then injected into the decoder via Adaptive Instance Normalization (AdaIN). This mechanism allows the network to modify the global characteristics of the face at multiple resolutions, from coarse $8 \times 8$ structures to fine $128 \times 128$ textures.
Spatial Information Transfer (SIT)
To compensate for the spatial invariance of style vectors, the SIT module preserves structural integrity. It extracts spatial details from the encoder and uses an attribute-difference-guided attention mechanism to fuse them into the decoder. This ensures that while the style (e.g., hair color) changes, the underlying structure (e.g., individual hair strands) remains sharp and consistent with the original identity.
Key Insights: Style as a Discriminative Feature
A pivotal finding of the research is that global style statistics are highly discriminative for facial attributes. By applying t-SNE visualization to the style features extracted at different stages, the authors demonstrated that these features naturally cluster according to specific attributes.
The visualization reveals an attribute hierarchy: different features become distinct at different layers. For instance, "Eyeglasses" are most discriminative at Stage 4, whereas gender-based features ("Male") are most distinct at Stage 3. By implementing skip connections across all six stages, SSCGAN effectively captures this hierarchical nature of facial attributes.
Results and Performance
Evaluated on the CelebA dataset, SSCGAN demonstrates superior performance over established models like StarGAN, AttGAN, and STGAN.
- Accuracy: SSCGAN achieved a classification accuracy of 94.21%, a substantial improvement over STGAN (83.21%) and RelGAN (83.59%).
- Fidelity: In terms of image quality, SSCGAN recorded a Frechet Inception Distance (FID) of 4.69, outperforming the previous state-of-the-art.
- Qualitative Quality: Visual inspections show that SSCGAN produces cleaner edits. When adding bangs, for example, the model avoids common artifacts like forehead ghosting or background distortion, which frequently plague models relying on local concatenation.
Conclusion
SSCGAN represents a significant step forward in facial attribute editing by treating attributes as global styles rather than local labels. By balancing global style shifts with local spatial preservation, it achieves high-fidelity results that respect both the target attribute and the original identity. While currently optimized for facial images, the underlying principles of style skip connections hold promise for broader applications in image-to-image translation and semantic segmentation.
Summary updated