Interpreting Galaxy Deblender GAN from the Discriminator's Perspective

Heyi Li, Yuewei Lin, Klaus Mueller, Wei Xu

Interpreting Galaxy Deblender GAN from the Discriminator's Perspective

Abstract

Generative Adversarial Networks (GANs) are increasingly used in astronomy to "deblend" overlapping galaxy images, a critical task for upcoming surveys like the Large Synoptic Survey Telescope (LSST). However, the lack of interpretability in GANs limits their scientific utility. We introduce Polarized-LRP, an explainable AI (XAI) technique designed to visualize the decision-making process of a GAN's Discriminator. Our method uncovered a "phantom boundary" artifact caused by zero-padding in data augmentation, which the model used as a shortcut instead of learning physical features. By diagnosing and correcting this, we demonstrate how XAI can transition from a curiosity to a vital debugging tool in scientific machine learning.

Introduction: The Blending Challenge

In modern astronomy, "blending" occurs when galaxies overlap along the line of sight. With the LSST expected to capture images where nearly 50% of galaxies are blended, automated deblending is essential. While GANs show promise, their "black box" nature creates a trust gap. Astronomers require models that are not only accurate but also physically grounded. When a GAN fails or produces artifacts, researchers often lack the tools to determine if the issue lies in the architecture, the training regime, or the underlying data.

Methodology: Polarized-LRP

To address this, we developed Polarized-LRP, an extension of Layer-wise Relevance Propagation (LRP). Standard LRP backpropagates relevance scores to highlight input pixels important for a classification. However, GAN Discriminators typically output a single scalar probability, making standard LRP difficult to apply.

Polarized-LRP solves this by splitting relevance into two distinct channels:

  1. Positive Relevance Maps: Identifying pixels that lead the Discriminator to classify an image as "Real."
  2. Negative Relevance Maps: Identifying pixels that contribute to a "Fake" classification.
Figure 2: Evolution of relevance maps and generated images across training stages.

Key Insights and Learning Dynamics

By applying Polarized-LRP throughout the training process, we observed several key phenomena regarding how the model matures.

The "Center-Out" Learning Process

Visualizations revealed that the Generator initially focuses on the "glaring spot" (the bright galactic center). This is likely driven by the initial style loss (VGG19). As training progresses, the Generator incrementally "apprehends" more complex outer structures, guided by the Discriminator's feedback.

The Discriminator as a Teacher

The Discriminator acts as more than a critic; it provides spatially specific feedback. Relevance maps often show "white rings" encircling the galaxy, indicating the specific regions where the Discriminator detects unrealistic transitions or artifacts. This feedback loop tells the Generator which specific pixels look unrealistic.

Figure 3: A detailed relevance map highlighting the Discriminator's focus on structural boundaries.

Discovery of the "Phantom Boundary"

The most significant finding was a rectangular outline appearing in positive relevance maps of ground truth images. Investigation revealed that the Discriminator was detecting sharp edges created by zero-padding during data augmentation. Instead of learning galactic physics, the model had found a "cheat code": identifying "real" images simply by the presence of these artificial boundaries.

Results and Debugging

The initial model replication achieved a Peak Signal-to-Noise Ratio (PSNR) of 33.47 dB and an SSIM of 0.89. However, the discovery of the "phantom boundary" allowed for targeted debugging. By replacing zero-padding with a random noise distribution modeled on actual sky statistics, we forced the Discriminator to ignore edge artifacts and focus on physical galactic features. This adjustment led to more robust and scientifically valid deblending results.

Conclusion and Future Work

Polarized-LRP provides a window into the adversarial game, allowing researchers to catch data-processing errors that would otherwise remain hidden. Future work will focus on extending this framework to interpret the Generator directly and applying these tools to more complex architectures like StyleGAN to build a comprehensive visual analytics system for astronomical AI.

Summary updated

Abstract

In large galaxy surveys it can be difficult to separate overlapping galaxies, a process called deblending. Generative adversarial networks (GANs) have shown great potential in addressing this fundamental problem. However, it remains a significant challenge to comprehend how the network works, which is particularly difficult for non-expert users. This research focuses on understanding the behaviors of one of the network’s major components, the Discriminator, which plays a vital role but is often overlooked. Specifically, we propose an enhanced Layer-wise Relevance Propagation (LRP) algorithm called Polarized-LRP. It generates a heatmap-based visualization highlighting the area in the input image that contributes to the network decision. It consists of two parts i.e. a positive contribution heatmap for the images classified as ground truth and a negative contribution heatmap for the ones classified as generated. As a use case, we have chosen the deblending of two overlapping galaxy images via a branched GAN model. Using the Galaxy Zoo dataset we demonstrate that our method clearly reveals the attention areas of the Discriminator to differentiate generated galaxy images from ground truth images, and outperforms the original LRP method. To connect the Discriminator’s impact on the Generator, we also visualize the attention shift of the Generator across the training process. An interesting result we have achieved is the detection of a problematic data augmentation procedure that would else have remained hidden. We find that our proposed method serves as a useful visual analytical tool for more effective training and a deeper understanding of GAN models.
BibTeX
@inproceedings{li2020interpreting,
  author = {Heyi Li and Yuewei Lin and Klaus Mueller and Wei Xu},
  title = {{Interpreting Galaxy Deblender GAN from the Discriminator's Perspective}},
  booktitle = {ISVC},
  year = {2020},
  url = {https://openreview.net/forum?id=l3-1WuY6nm3},
}
Figure