Introduction
Generative Adversarial Networks (GANs), particularly StyleGAN2, have revolutionized image synthesis. Beyond generating high-fidelity images, their latent spaces contain "directions" that correspond to semantic attributes like age, hair color, or lighting. However, discovering these directions typically requires either extensive manual labeling (supervised) or results in highly redundant "knobs" (unsupervised).
The Redundancy Challenge
Current unsupervised methods, such as PCA-based GANSpace, often fail to capture the full breadth of a model's capabilities. In a face model, for instance, an automated search might return dozens of directions that all modify the background while completely missing subtle features like ear shape or clothing details. This redundancy stems from the high dimensionality of StyleGAN2’s StyleSpace ($S$-space), which contains over 9,000 channels. For example, the authors found over 300 channels dedicated to hair and 180 to background/ears; without a diversity constraint, an automated search will likely return multiple redundant ways to change the same attribute.
A Submodular Framework for Discovery
To address this, researchers proposed a novel framework that treats direction discovery as a combinatorial optimization problem using submodular functions. This approach aims to pick a small, representative subset of channels that covers as many unique types of edits as possible without repetition.
1. Quantifying and Clustering Effects
The method first measures the perceptual impact of perturbing individual channels using two metrics: LPIPS (to measure the magnitude of perceptual change) and SSIM (to measure structural similarity). Channels that affect similar image regions (e.g., all channels modifying the nose) are grouped into clusters using Agglomerative Clustering based on their SSIM scores.
2. The Submodular Objective
The core of the framework is a mathematical objective function designed to reward two properties:
- Coverage: Selecting channels that are highly representative of the overall latent space.
- Diversity: Applying a "diminishing returns" logic. If a channel from a specific cluster (like "Hair") is already selected, the marginal reward for picking another from the same cluster is significantly reduced (using a log-sum function).
3. Greedy Optimization
Because the objective function is submodular, a simple "greedy" algorithm—selecting the best remaining channel one by one—is mathematically guaranteed to provide a solution very close to the theoretical optimum.
Key Results and Insights
The framework was evaluated across a wide variety of domains, including FFHQ (faces), Metfaces (portraits), AFHQ Cats, LSUN Cars, and a custom Fashion dataset.
- Superior Disentanglement: By operating in $S$-space rather than the more common $W$ or $W+$ spaces, the method achieves much cleaner edits. In a study with 25 participants, this method scored 4.32/5.0 for disentanglement, significantly outperforming GANSpace (2.46) and SeFa (2.91).
- Uncovering Hidden Attributes: Unlike supervised methods, this framework discovered attributes that weren't even in the training labels, such as specific "clothing patterns" or "neck types" in fashion models.
- Efficiency: The automated nature of the submodular selection removes the need for human-in-the-loop exploration of thousands of channels, effectively avoiding the "redundancy trap."
Conclusion and Future Work
By framing latent direction discovery through the lens of submodular optimization, this work provides a powerful tool for exploring and utilizing pre-trained GANs. While currently optimized for StyleGAN2, the authors suggest the framework could be extended to other architectures and real-image editing via GAN inversion. It successfully navigates the trade-off between manual supervision and redundant unsupervised discovery, offering a diverse and disentangled set of controls for high-quality image manipulation.
Summary updated