Abstract
Recent advances in generative adversarial networks (GANs), particularly StyleGAN, have enabled significant progress in photorealistic image generation. However, these models often lack disentanglement—the ability to edit specific attributes independently without affecting others. We propose a lightweight auto-encoder architecture that re-organizes the StyleGAN latent space into a compact, semantic representation where each axis corresponds to a specific, decorrelated attribute. By operating within a compressed latent subspace derived via Principal Component Analysis (PCA), our method reduces parameter complexity and achieves remarkably short training times (~45 minutes). Quantitative and qualitative evaluations demonstrate that our approach provides superior disentanglement and identity preservation compared to existing state-of-the-art methods.
Introduction
Generative Adversarial Networks (GANs) have set the standard for high-fidelity image synthesis. Despite their power, navigating their internal latent spaces for precise image editing remains a challenge. In StyleGAN, the latent space $W$ is often highly entangled; for instance, attempting to increase a subject's "age" might inadvertently alter their gender or background.
To be effective for practical applications, an image editing framework must satisfy three primary criteria:
- Disentanglement: Modifications to one attribute should not leak into others.
- Controllability: Attributes should map to intuitive, linear axes (e.g., sliders).
- Fidelity: The edited image must maintain the original subject's identity and overall visual quality.
While existing methods attempt to find directions in the latent space for editing, they often require complex optimization or utilize expanded spaces (like $W+$) that can introduce unnatural artifacts. Our work addresses these issues by introducing a semantic auto-encoder that enforces independence between attributes directly within a compressed latent space.
Methodology
PCA-Based Compression
The latent space $W$ of StyleGAN2 is typically 512-dimensional and over-parametrized. We observed that a significant portion of the variance is captured by a small number of dimensions. By applying Principal Component Analysis (PCA), we identified that just 60 dimensions ($W_{PCA}$) capture approximately 80% of the variance. Restricting editing to this "safe" subspace helps prevent the model from entering low-variance regions that typically cause "ghosting" or visual artifacts.
Semantic Auto-encoder Architecture
We introduce an encoder $E$ and a decoder $D$ that sit atop StyleGAN2. The encoder maps $W_{PCA}$ to a new semantic space $C$. In this space, the first $K$ dimensions are explicitly mapped to specific attributes (e.g., dimension 1 for smile, dimension 2 for age). The decoder then maps these semantic coordinates back to the $W_{PCA}$ space for image generation.
Training and Loss Functions
The model is trained entirely within the latent space, avoiding the need for expensive image generation during the training loop. We utilize a composite loss function:
- Reconstruction Loss: Ensures that the auto-encoder can accurately reconstruct the original latent vector when no edits are applied.
- Attribute Loss: Uses labels from a pre-trained classifier to force specific latent axes to correspond to specific physical attributes.
- Correlation Loss: This is the critical component for disentanglement. It penalizes correlations between different axes in the $C$ space, forcing the model to find independent directions for each attribute.
Experimental Setup
We utilized a dataset of 200,000 synthetic latent vectors generated by StyleGAN2. To provide ground-truth labels for training, we employed an EfficientNet-B0 classifier pre-trained on the CelebA dataset, covering 40 distinct facial attributes. The training process is highly efficient, requiring only 45 minutes on a single Nvidia RTX 3090 GPU.
Results and Discussion
Disentanglement and Fidelity
Quantitative analysis using attribute variation matrices shows that our method significantly reduces off-diagonal interference compared to competitors like InterfaceGAN and GANSpace. This indicates that our "Age" slider, for example, has a much lower impact on "Gender" or "Expression" than previous approaches.
Furthermore, our method achieves higher Identity Preservation scores (measured via cosine distance of face embeddings) and superior Fréchet Inception Distance (FID) scores. By staying within the native $W$ space and utilizing PCA compression, we ensure that edited images remain within the distribution of the original generator, preserving realism.
Qualitative Performance
Visual results confirm the quantitative findings. Edits performed using our semantic axes result in smooth, natural transitions. The "slider" philosophy—mapping attributes directly to independent axes—transforms complex high-dimensional navigation into a simple linear adjustment suitable for consumer-grade photo editing software.
Conclusion and Future Work
We have presented a simple yet effective method for creating a compact and semantic latent space for StyleGAN. By training a decorrelated auto-encoder on PCA-compressed latent vectors, we achieve high-quality, disentangled edits with minimal computational overhead.
Future work includes expanding this approach to more granular attributes beyond the 40 CelebA categories and exploring its application in non-facial domains. While our reliance on pre-trained classifiers introduces potential biases, the efficiency and fidelity of the current framework provide a robust foundation for controllable generative modeling.
Summary updated