Abstract
Generative Adversarial Networks (GANs) have become a cornerstone of medical image-to-image translation, yet their tendency to "hallucinate" anatomical features poses a significant risk in clinical settings. This paper proposes a "stress test" for GANs by injecting noise into their latent representations. By measuring output fluctuations—specifically variance and mutual information—researchers can quantify uncertainty and identify unreliable features. Our findings show that this method not only flags corrupted inputs but also improves synthesis quality when integrated into the training process.
Introduction
In the domain of medical imaging, GANs are frequently employed to translate between modalities, such as synthesizing a CT scan from an MRI. However, the black-box nature of these models leads to a critical issue of trust: models may omit real pathology or invent non-existent structures to satisfy learned patterns. Existing uncertainty estimation techniques often require architectural overhauls or computationally expensive optimizations. This research introduces a "plug-and-play" noise injection technique to evaluate the robustness of latent representations, providing a metric for the reliability of generated outputs.
Methodology: The Latent Stress Test
The core of the proposed approach is a noise injection technique targeting the GAN's bottleneck—the latent representation ($h$). The process involves three steps:
- Corruption: An input image is encoded into a latent code $h$. Gaussian noise is then added to create $k$ variations of this code.
- Generation: These $k$ noisy codes are passed through the decoder, resulting in a set of slightly different output images.
- Quantification: Two primary metrics are derived:
- Variance ($\gamma$): Measures the pixel-wise difference among the $k$ outputs. High variance indicates the model is "guessing" in specific regions.
- Mutual Information ($\delta$): Acts as a confidence score by measuring how much information is preserved between the original output and its noisy counterparts.
Key Insights: Visualizing the Unknown
A primary advantage of this method is the ability to generate "uncertainty heatmaps." By intentionally perturbing the input image (e.g., blacking out a patch), the authors demonstrated that the resulting variance and confidence scores accurately flag these regions as highly uncertain.
In a clinical workflow, these heatmaps could serve as an automated "red flag" system, alerting radiologists to specific areas in a synthetic image that may be hallucinations. Furthermore, the study found that injecting a small amount of noise ($\alpha = 0.5$) during the training phase acts as a regularizer. This forces the encoder to learn more stable and meaningful latent codes, ultimately improving the Fréchet Inception Distance (FID) scores of the generated images.
Key Results and Quantitative Validation
The effectiveness of the metrics was validated using the CHAOS19 and LiTS datasets. In a "sanity check" involving CT to T1 translation, the variance ($\gamma$) increased from 0.0567 for original images to 0.0676 for perturbed images. Simultaneously, the confidence score ($\delta$) showed a marked decrease, confirming that the metrics are sensitive to input errors.
Beyond visual quality, the researchers explored the correlation between latent robustness and downstream task performance. For successful translations, the proposed confidence score ($\delta$) correlated significantly (often $>0.5$) with the DICE coefficient of a subsequent liver segmentation task. This suggests that the confidence score can serve as a proxy for whether a synthetic image is of sufficient quality for further automated analysis.
Conclusion and Future Directions
While noise injection improves overall image quality and provides a robust uncertainty metric, it is not a panacea. The authors noted that increased robustness during training can sometimes lead to a loss of fine-grained detail necessary for precise boundary detection in segmentation tasks. Additionally, the correlation between confidence and accuracy is less reliable when the base image quality is very low. Future research will investigate targeted adversarial attacks to further probe the limits of these latent representations and validate the approach in broader clinical scenarios.
Summary updated