Introduction
Understanding galaxy morphology—the study of shape and structure—is fundamental to astrophysics, offering insights into evolutionary history and cosmic dynamics. With modern surveys producing millions of images, Convolutional Neural Networks (CNNs) have become the standard for automated classification. However, the "black-box" nature of these models often obscures the physical features driving their decisions, limiting scientific trust. This work addresses this gap by generating visual counterfactual explanations (CEs): synthetic images that illustrate the minimal, semantically meaningful changes needed to flip a model's prediction from one morphological class to another.
Methodology
The proposed architecture replaces the standard classification head with an Invertible Neural Network (INN) to enable bidirectional mapping between image features and class representations.
The process involves three core components:
- Latent Disentanglement: An encoder maps the input image into a latent space split into a class-dependent vector ($z_1$) and a class-independent vector ($z_2$). This ensures that background details are preserved during counterfactual generation.
- Invertible Normalizing Flow: The $z_1$ vector is processed through an invertible flow ($F$) that maps it to a Gaussian Mixture Model (GMM). Classification is performed based on the proximity to cluster centers in this hidden space.
- Counterfactual Synthesis: To generate an explanation, the model shifts a galaxy's hidden representation across a decision boundary into a target class's cluster. The inverse flow ($F^{-1}$) then produces a modified $z_1^{cf}$, which is combined with the original $z_2$ and passed through a decoder to reconstruct the counterfactual image.
The model is trained using a composite loss function including perceptual reconstruction loss, Maximum Mean Discrepancy (MMD) for latent continuity, and an Information Bottleneck loss to ensure $z_1$ contains only essential classification features.
Experimental Results
The model was evaluated on the Galaxy10 DECaLS dataset, comprising 17,736 images across 10 morphological classes.
Classification Performance
The framework achieves an overall accuracy of approximately 80%. While most classes perform well (70-90%), the model struggles with "Disturbed" galaxies (41% accuracy), likely due to their chaotic and diffuse nature which prevents clean clustering in the latent space.
Counterfactual Realism and Interpretability
The generated counterfactuals are highly realistic, maintaining a Structural Similarity Index Measure (SSIM) of 0.96 relative to the original images.
As shown in the figure above, the model successfully isolates and modifies class-defining features. For instance, converting a "round smooth" galaxy to a "cigar-shaped" one involves a clear elongation of the central light distribution while keeping the background noise identical. This demonstrates effective feature disentanglement.
Discussion and Limitations
A key insight is the model's ability to visualize its internal decision boundaries. However, the use of an Information Bottleneck introduces a trade-off: the compression required for robust classification sometimes discards fine-grained details like delicate spiral arms. Consequently, counterfactuals between similar spiral sub-types can appear visually indistinguishable.
Future work aims to refine the architecture to better capture high-frequency morphological structures. Additionally, the sensitivity of the latent distributions to mislabeled data suggests a potential secondary use-case: automatically identifying and flagging noisy labels in large-scale astronomical datasets.
Conclusion
By combining invertible flows with latent space splitting, this framework provides a powerful tool for interpretable galaxy classification. It moves beyond simple accuracy metrics, offering astrophysicists visual evidence of the morphological features that drive machine learning predictions.
Summary updated