Introduction
While StyleGAN is renowned for producing hyper-realistic faces, the original version (StyleGAN1) frequently generates unsightly circular blobs, often referred to as "water droplets." These artifacts are more than just a visual nuisance; they represent a fundamental instability in the generator's architecture.
Previous work, such as StyleGAN2, addressed this using weight demodulation. However, the community lacked a systematic understanding of why these artifacts formed, why they were specifically circular, and why they persisted through the network's layers. This paper provides a mechanistic explanation and proposes a simpler architectural fix.
Methods
The authors employ a mix of visualization, ablation studies, and mathematical analysis to diagnose the issue. Using the GANDissect framework, they identified specific "hidden units" (neurons) in the convolutional layers where ablating just one high-activation unit could move or alter the artifact, proving these spots are tied to specific feature map activations.
Pixel-Instance Normalization (PIN)
To solve the problem, the authors propose a new normalization layer that is a learnable, weighted combination of Pixel Normalization (PN) and Instance Normalization (IN):
$$y = \rho \cdot y^P + (1 - \rho) \cdot y^I$$
Here, $\rho$ is a trainable parameter. This allows the network to decide, channel-by-channel, whether it needs the style-transferring power of IN or the magnitude-limiting stability of PN. This layer replaces the standard AdaIN (which relies on IN) to maintain high visual fidelity without the "droplets."
Key Insights and Ideas
The "Water Droplet" as a Runaway Signal
The artifacts are triggered by random noise injected into the generator. Because Instance Normalization (IN) normalizes based on the mean and variance of the entire feature map, a single "spike" in a sparse area forces the rest of the map to flatten, making that spike even more dominant in the subsequent layer.
Why are they circular?
The authors attribute the shape to the $3 \times 3$ convolutional kernels. A pixel at the center of a "spike" has more high-magnitude neighbors than those at the edges, creating a gradient that naturally settles into a circular, ring-like boundary after multiple layers.
Learned Behavior of PIN
The network's preference for normalization types shifts through the architecture. Early layers (low resolution) prefer Instance Normalization to establish broad styles, while later layers (high resolution) shift toward Pixel Normalization to prevent magnitudes from escalating into artifacts. The learned $\rho$ values confirm this trend, moving from IN for "heavy lifting" to PN for stability.
Key Results
- Artifact Removal: The PIN layer successfully removed circular artifacts across various seeds and latent vectors.
- Quality Preservation: PIN achieves a Fréchet Inception Distance (FID) that maintains the high visual fidelity of StyleGAN. Simply using PN everywhere removes artifacts but results in "flat," lower-quality images.
- Architecture Trends: Analysis of learned parameters shows the network uses IN in $4 \times 4$ and $8 \times 8$ layers but switches to PN in $256 \times 256$ to $1024 \times 1024$ layers.
Conclusion
While weight demodulation has become the standard fix, PIN offers a simpler, "plug-and-play" alternative for existing architectures. This systematic analysis—combining GANDissect with normalization theory—provides a blueprint for debugging unusual artifacts in other emerging generative architectures like Transformers or Diffusion models.
Summary updated