Abstract
StyleGAN models often produce visual artifacts due to a phenomenon termed "Feature Proliferation," where high-magnitude features dominate and multiply across layers. While the standard "truncation trick" mitigates these artifacts by pulling latent codes toward the mean, it significantly reduces image diversity. This paper introduces a surgical feature rescaling method that identifies and modulates risky features in early layers. This approach effectively "cures" artifacts while preserving fine details and diversity, outperforming traditional truncation across multiple benchmarks.
Introduction
StyleGAN is a titan in the world of image synthesis, but it isn't perfect. It often produces "artifacts"—strange, non-realistic blobs or distortions in generated images. To fix this, researchers typically use the truncation trick, a post-processing step that pulls the latent code toward the average.
While effective at improving quality, the truncation trick is a blunt instrument. It works in a high-level latent space, meaning it often "throws the baby out with the bathwater" by sacrificing image diversity and erasing unique details (like specific facial hair or accessories) just to get rid of the glitches. The authors sought to find a more "fine-grained" way to fix these artifacts without losing the variety that makes GANs useful.
Methods: Identifying and Treating the "Cancer"
The authors' approach moves the intervention from the high-level latent space down into the feature space of the generator. They identify a process called Feature Domination, where a small number of features with abnormally large values dominate the weighted sum of a neuron. When these dominant features reproduce and increase in ratio through subsequent layers, it is termed Feature Proliferation.
The Statistical Heuristic
To find these "risky" features, the researchers look for feature maps that deviate significantly from the mean of their distribution across the training set. They use a specific formula ($r_{l,j}$) to calculate the "risk" of a feature based on its mean and standard deviation.
Surgical Feature Rescaling
Instead of normalizing the entire output, the method only rescales the identified risky features. By dividing these features by a factor proportional to their risk and a scaling hyperparameter ($p$), they dampen the "cancerous" growth before it can ruin the final image. This method follows a "no retraining" philosophy, meaning it can be applied to existing, pre-trained StyleGAN models without the massive computational cost of training from scratch.
Key Results
The proposed method was evaluated using standard high-resolution datasets including FFHQ (faces), MetFace (portraits), and AFHQ-Cat.
Detail Preservation and Diversity
Qualitatively, the method shines where the truncation trick fails. While truncation often removes eyeglasses or beards to fix artifacts, the rescaling method successfully removes the artifacts while keeping fine facial details intact. Quantitative analysis using Precision and Recall showed that the proposed method achieves high image quality with much less sacrifice to diversity (recall) compared to the truncation trick.
Quantitative Superiority
The method consistently outperformed the truncation trick across multiple metrics, including FID (Fréchet Inception Distance), PSNR, and SSIM. For example, on StyleGAN2 MetFace, the FID improved from 23.02 (truncation) to 17.51 (ours). Furthermore, the method remains fully compatible with existing StyleGAN operations like latent space interpolation and image editing.
Conclusion and Future Work
The authors traced the root cause of Feature Proliferation to a statistical flaw: StyleGAN models assume input activations are independent and identically distributed (i.i.d.) with unit variance, which is not true in practice. By intervening surgically in the feature space, they target the specific channels causing the problem while leaving the rest of the image's "DNA" intact.
While highly effective, the heuristic might still occasionally dampen useful features or fail to catch extremely subtle artifacts. Future work will investigate if this "cancerous" proliferation is a universal problem in other deep learning architectures beyond StyleGAN.
Summary updated