DiffExplainer: Unveiling Black Box Models Via Counterfactual Generation

Yingying Fang, Shuang Wu, Zihao Jin, Shiyi Wang, Caiwen Xu, Simon Walsh, Guang Yang

Abstract

Deep learning models have demonstrated significant potential in medical prognosis, yet their "black box" nature remains a primary barrier to clinical adoption. Traditional explainability methods, such as attribution heatmaps, often lack the granularity required to identify subtle textural biomarkers. This paper presents DiffExplainer, a novel framework that utilizes Diffusion Autoencoders to generate high-fidelity counterfactual medical images. By manipulating latent representations to flip model predictions, DiffExplainer provides clinicians with precise difference maps that highlight the specific anatomical features driving AI decisions. Our results demonstrate superior image reconstruction and high fidelity to the underlying black-box models, offering a more intuitive and actionable diagnostic tool.

Introduction

In the management of chronic conditions like Fibrotic Lung Disease (FLD), predicting patient mortality is crucial for timely intervention. While deep learning classifiers can achieve high predictive accuracy, they fail to provide the "why" behind a high-risk score. Existing Explainable AI (XAI) techniques like Grad-CAM often produce blurry regions of interest that do not distinguish between healthy tissue and subtle pathological textures. Furthermore, previous generative approaches using GANs often struggle with reconstruction fidelity, introducing artifacts that obscure the very features they aim to explain. DiffExplainer addresses these challenges by leveraging the generative power of Diffusion Models to create near-lossless reconstructions and meaningful counterfactual scenarios.

Methodology

The DiffExplainer framework operates through a three-stage pipeline designed to bridge the gap between complex model logic and visual interpretation.

The comprehensive workflow of DiffExplainer, including AutoEncoder development, Agent Classifier training, and the three-step feature identification process.

1. Diffusion Autoencoder (DiffAE)

Unlike standard diffusion models, DiffExplainer employs a Diffusion Autoencoder. This architecture uses an encoder ($E$) to map an input image ($x_0$) to a semantically meaningful latent vector ($z$). This vector captures the essential features of the scan, allowing the model to reconstruct the original image with near-lossless quality. This is critical in medical imaging, where preserving fine details like airways and vascular structures is paramount.

2. Teacher-Student Learning (The Agent Model)

To explain a complex "Black Box" teacher model ($C$), a shallow "Agent" model ($A$) is trained to mimic its behavior. The Agent learns to predict clinical outcomes (e.g., Death vs. Survival) based solely on the latent vector $z$ extracted by the DiffAE. This ensures that the explanations generated are representative of the logic used by the primary classifier.

3. Counterfactual Generation and Feature Identification

To generate an explanation, the system calculates the gradient of the Agent’s output with respect to the latent vector. By "nudging" the vector in a direction that either increases or decreases the predicted risk (controlled by a step size $\alpha$), a manipulated latent vector is created. This vector is then decoded back into a "counterfactual" image. The final Difference Map is produced by subtracting the original image from the counterfactual, pinpointing the exact physical features the AI prioritized.

Experimental Setup

The framework was evaluated using datasets focused on Fibrotic Lung Disease:

  • OSIC (Open Source Imaging Consortium): Utilized for the primary training of the Diffusion Autoencoder and the Agent model.
  • In-house Australian Dataset: An independent cohort of CT scans from 501 patients (43 deceased within one year, 458 survivors) used for rigorous evaluation of the model's explanatory power.

Results and Discussion

DiffExplainer demonstrated significant technical and qualitative advantages over baseline methods. The Diffusion Autoencoder achieved a PSNR of 35.28 and an SSIM of 0.99, drastically outperforming GAN-based alternatives in preserving medical textures. The Agent model successfully mimicked the black-box teacher with an AUC of 0.99 and 96% accuracy, confirming that the generated counterfactuals are faithful to the original model's decision-making process.

Visual results showing counterfactual generation for varying alpha values. The difference maps highlight specific textural changes in the lung tissue as the mortality risk increases or decreases.

Qualitatively, DiffExplainer provides a level of granularity previously unavailable. As shown in the visual results, the framework can pinpoint specific anatomical shifts—such as increased lung density or the progression of textural scarring—rather than just highlighting broad regions. This "what-if" capability allows clinicians to see exactly how much a specific feature would need to change to alter a patient's prognosis, making the AI's output far more actionable.

Conclusion

DiffExplainer represents a significant step forward in making medical AI transparent. By combining the reconstruction fidelity of diffusion models with a teacher-student learning paradigm, it provides high-resolution, semantically meaningful explanations. Future work will focus on reducing the computational intensity of the iterative sampling process and conducting formal clinical validation studies with radiologists to confirm the alignment of AI-identified biomarkers with established medical knowledge.

Summary updated

BibTeX
@inproceedings{fang2024diffexplainer,
  author = {Yingying Fang and Shuang Wu and Zihao Jin and Shiyi Wang and Caiwen Xu and Simon Walsh and Guang Yang},
  title = {{DiffExplainer: Unveiling Black Box Models Via Counterfactual Generation}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2024},
  doi = {10.1007/978-3-031-72117-5\_20},
}
Figure