Systematic Analysis and Removal of Circular Artifacts for StyleGAN

Way Tan, Bihan Wen, Xulei YangarXiv 2021 · arXiv

Introduction

While StyleGAN is renowned for producing hyper-realistic faces, the original version (StyleGAN1) frequently generates unsightly circular blobs, often referred to as "water droplets." These artifacts are more than just a visual nuisance; they represent a fundamental instability in the generator's architecture.

Examples of circular artifacts (water droplets) in images generated by StyleGAN1, highlighted by red circles.

Previous work, such as StyleGAN2, addressed this using weight demodulation. However, the community lacked a systematic understanding of why these artifacts formed, why they were specifically circular, and why they persisted through the network's layers. This paper provides a mechanistic explanation and proposes a simpler architectural fix.

Methods

The authors employ a mix of visualization, ablation studies, and mathematical analysis to diagnose the issue. Using the GANDissect framework, they identified specific "hidden units" (neurons) in the convolutional layers where ablating just one high-activation unit could move or alter the artifact, proving these spots are tied to specific feature map activations.

Pixel-Instance Normalization (PIN)

To solve the problem, the authors propose a new normalization layer that is a learnable, weighted combination of Pixel Normalization (PN) and Instance Normalization (IN):

$$y = \rho \cdot y^P + (1 - \rho) \cdot y^I$$

Here, $\rho$ is a trainable parameter. This allows the network to decide, channel-by-channel, whether it needs the style-transferring power of IN or the magnitude-limiting stability of PN. This layer replaces the standard AdaIN (which relies on IN) to maintain high visual fidelity without the "droplets."

Key Insights and Ideas

The "Water Droplet" as a Runaway Signal

The artifacts are triggered by random noise injected into the generator. Because Instance Normalization (IN) normalizes based on the mean and variance of the entire feature map, a single "spike" in a sparse area forces the rest of the map to flatten, making that spike even more dominant in the subsequent layer.

Heatmaps of feature maps showing how sparse, high-magnitude activations (spikes) correspond to the location of artifacts in the generated image.

Why are they circular?

The authors attribute the shape to the $3 \times 3$ convolutional kernels. A pixel at the center of a "spike" has more high-magnitude neighbors than those at the edges, creating a gradient that naturally settles into a circular, ring-like boundary after multiple layers.

Learned Behavior of PIN

The network's preference for normalization types shifts through the architecture. Early layers (low resolution) prefer Instance Normalization to establish broad styles, while later layers (high resolution) shift toward Pixel Normalization to prevent magnitudes from escalating into artifacts. The learned $\rho$ values confirm this trend, moving from IN for "heavy lifting" to PN for stability.

Key Results

  • Artifact Removal: The PIN layer successfully removed circular artifacts across various seeds and latent vectors.
  • Quality Preservation: PIN achieves a Fréchet Inception Distance (FID) that maintains the high visual fidelity of StyleGAN. Simply using PN everywhere removes artifacts but results in "flat," lower-quality images.
  • Architecture Trends: Analysis of learned parameters shows the network uses IN in $4 \times 4$ and $8 \times 8$ layers but switches to PN in $256 \times 256$ to $1024 \times 1024$ layers.

Conclusion

While weight demodulation has become the standard fix, PIN offers a simpler, "plug-and-play" alternative for existing architectures. This systematic analysis—combining GANDissect with normalization theory—provides a blueprint for debugging unusual artifacts in other emerging generative architectures like Transformers or Diffusion models.

Summary updated

Abstract

StyleGAN is one of the state-of-the-art image generators which is well-known for synthesizing high-resolution and hyper-realistic face images. Though images generated by vanilla StyleGAN model are visually appealing, they sometimes contain prominent circular artifacts which severely degrade the quality of generated images. In this work, we provide a systematic investigation on how those circular artifacts are formed by studying the functionalities of different stages of vanilla StyleGAN architecture, with both mechanism analysis and extensive experiments. The key modules of vanilla StyleGAN that promote such undesired artifacts are highlighted. Our investigation also explains why the artifacts are usually circular, relatively small and rarely split into 2 or more parts. Besides, we propose a simple yet effective solution to remove the prominent circular artifacts for vanilla StyleGAN, by applying a novel pixel-instance normalization (PIN) layer.
BibTeX
@misc{tan2021systematic,
  author = {Way Tan and Bihan Wen and Xulei Yang},
  title = {{Systematic Analysis and Removal of Circular Artifacts for StyleGAN}},
  year = {2021},
  eprint = {2103.01090},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2103.01090},
}
Figure