Micro-Ct Synthesis and Inner Ear Super Resolution via Generative Adversarial Networks and Bayesian Inference

Hongwei Li, Rameshwara G. N. Prasad, Anjany Sekuboyina, Chen Niu, Siwei Bai, Werner Hemmert, Bjoern H. Menze

Introduction

Precision is paramount in otologic surgery, particularly for procedures like cochlear implantation where surgeons must navigate delicate inner ear structures. While clinical Computed Tomography (CT) is the standard for preoperative planning, its resolution often fails to capture the finest anatomical details. Micro-CT provides the necessary resolution but is unsuitable for live patients due to high radiation doses and hardware constraints. Super-Resolution (SR) techniques offer a potential bridge, but traditional supervised models require perfectly aligned paired datasets—clinical CT and Micro-CT of the same subject—which are rarely available. Furthermore, models trained on synthetic downsampled data often struggle with the complex noise profiles of real clinical scans.

Methods

To address these challenges, the researchers developed a framework that utilizes unpaired image-to-image translation. The core of the system is a Cycle-Consistent Generative Adversarial Network (CycleGAN), which learns to map the distribution of clinical CT scans to that of Micro-CT scans without requiring pixel-perfect alignment.

Proposed framework combining CycleGAN for unpaired translation and Bayesian inference for uncertainty estimation.

The framework incorporates two additional innovations:

  1. Bayesian Inference: By employing Monte Carlo Dropout, the model generates multiple outputs for a single input. The variance across these outputs is used to create an uncertainty map, highlighting regions where the model's predictions are less certain, thus providing a safety mechanism against "hallucinated" details.
  2. Structural Evaluation: To quantify performance on unpaired data, the authors introduced Maximum Hu Moments (m-HuM). This metric measures shape similarity in a way that is invariant to rotation and scale, bypassing the need for image registration.

Key Results

The proposed method demonstrated significant improvements over state-of-the-art supervised approaches like U-Net and SR-GAN.

Quantitative Performance

In evaluations using the m-HuM metric (where lower values indicate better shape similarity), the proposed Bayesian GAN achieved a score of 0.28, significantly outperforming SR-GAN (1.29) and U-Net (3.12). This indicates a much higher fidelity to the underlying anatomical shapes.

Qualitative and Expert Evaluation

Blind visual assessments by three experienced otolaryngology researchers consistently ranked the proposed method highest in terms of image quality, resolution, and structural preservation.

Qualitative comparison of super-resolution methods and the generated uncertainty maps.

As shown in the visual results, the Bayesian GAN maintains sharp, anatomically correct boundaries of the bony structures of the inner ear. In contrast, other methods often introduced blurry edges or artifacts. Notably, the uncertainty maps were found to highlight the boundaries of the temporal bone, suggesting their potential utility in automated segmentation tasks.

Discussion and Conclusion

By leveraging unpaired training and Bayesian uncertainty, this approach overcomes the primary barriers to applying super-resolution in clinical settings. The model's ability to learn from real-world noise distributions makes it more robust than supervised alternatives. While the computational cost of Bayesian inference is higher, the added reliability and the diagnostic value of uncertainty maps are significant advantages for medical applications. Future work will focus on optimizing processing times and validating the impact of these synthesized high-resolution images on actual surgical outcomes.

Summary updated

Abstract

Existing medical image super-resolution methods rely on pairs of low- and high-resolution images to learn a mapping in a fully supervised manner. However, such image pairs are often not available in clinical practice. In this paper, we address super resolution problem in a real-world scenario using unpaired data and synthesize linearly eight times higher resolved Micro-CT images of temporal bone structure embedded in the inner ear. We explore cycle-consistency generative adversarial networks for super-resolution and equip the model with Bayesian inference. We further introduce Hu Moments distance as the evaluation metric to quantify the shape of the temporal bone. We evaluate our method on a public inner ear CT dataset and have seen both visual and quantitative improvement over state-of-the-art supervised deep-learning based methods. Further, we conduct a multi-rater visual evaluation experiment and find that three inner-ear researchers consistently rate our method highest quality scores among three methods. Furthermore, we are able to quantify uncertainty in the unpaired translation task and the uncertainty map can provide structural information of the temporal bone.
BibTeX
@inproceedings{li2021micro,
  author = {Hongwei Li and Rameshwara G. N. Prasad and Anjany Sekuboyina and Chen Niu and Siwei Bai and Werner Hemmert and Bjoern H. Menze},
  title = {{Micro-Ct Synthesis and Inner Ear Super Resolution via Generative Adversarial Networks and Bayesian Inference}},
  booktitle = {ISBI},
  year = {2021},
  doi = {10.1109/isbi48211.2021.9434061},
}

Loading related content...

Figure