Tunable image quality control of 3-D ultrasound using switchable CycleGAN

Jaeyoung Huh, Shujaat Khan, Sungjin Choi, Dongkuk Shin, Jeong Eun Lee, Eun Sun Lee, Jong Chul YeMedical Image Analysis 2021 · arXiv · OpenReview

Abstract

In contrast to 2-D ultrasound (US) for uniaxial plane imaging, a 3-D US imaging system can visualize a volume along three axial planes. This allows for a full view of the anatomy, which is useful for gynecological (GYN) and obstetrical (OB) applications. Unfortunately, 3-D US has an inherent limitation in resolution compared to 2-D US. In the case of 3-D US with a 3-D mechanical probe, for example, the image quality is comparable along the beam direction, but significant deterioration in image quality is often observed in the other two axial image planes. To address this, we propose a novel unsupervised deep learning approach to improve 3-D US image quality. Using unmatched high-quality 2-D US images as a reference, we trained a switchable CycleGAN architecture so that every mapping plane in 3-D US can learn the image quality of 2-D US images. Our network provides real-time control of the image enhancement level based on user preference, ideal for a user-centric scanner setup.

Introduction: The Resolution Gap in 3-D Ultrasound

While 3-D ultrasound is invaluable for visualizing complex anatomy in obstetrics and gynecology, it suffers from a significant "resolution gap" compared to traditional 2-D imaging. In a 3-D scan, the "A-plane" (axial-lateral) typically maintains high quality, but the "B" and "C" planes (elevation-lateral) often suffer from blurring and artifacts due to mechanical scanning processes or sparse sensor arrays.

Comparison of 2-D and 3-D ultrasound imaging pipelines, illustrating the multi-planar nature of 3-D scans and the resulting resolution challenges in the B and C planes.

Improving this quality is challenging because it is physically impossible to acquire "perfect" 3-D scans to serve as ground truth for supervised training. Furthermore, clinical preferences vary; one radiologist might prefer a sharp, high-contrast image, while another might find such processing unrealistic or prone to artifacts.

Methodology: Switchable CycleGAN

The proposed solution utilizes a Switchable CycleGAN architecture designed for unsupervised learning. By treating high resolution as a "style," the model learns the distribution of high-quality 2-D images and applies that style to degraded 3-D planes.

Key Technical Innovations:

  • Unsupervised Learning: The framework maps blurry B and C planes into the domain of high-quality 2-D A-planes without requiring paired data.
  • Adaptive Instance Normalization (AdaIN): Instead of multiple generators, a single generator is controlled by AdaIN codes to manage different enhancement styles.
  • The "Alpha" ($α$) Knob: Users can interpolate between an "identity" code (raw image) and an "enhancement" code (full 2-D quality mapping). By sliding a value from 0.0 to 1.0, clinicians can find their preferred balance of enhancement in real-time.
  • Spatial Control: The system allows for localized enhancement by applying different codes to specific parts of an image using a mask.

Results and Clinical Validation

The study utilized a robust clinical dataset comprising 208 gynecology (GYN) and 115 obstetrics (OB) volumes, with nearly 15,000 high-quality 2-D images used as the target domain.

Clinical Insights:

  • Expert Assessment: Blinded evaluations by a board-certified radiologist confirmed statistically significant improvements in contrast and overall quality.
  • Artifact Suppression: The model successfully sharpened follicle boundaries in GYN scans and suppressed common "side-lobe" noise.
  • Obstetric Clarity: In fetal scans, internal head bone structures became significantly more visible, addressing a common challenge in 3-D rendering.
  • Real-Time Performance: The system achieved a processing speed of approximately 0.14 seconds per frame on standard hardware (NVIDIA GTX 1080 Ti), making it viable for live clinical use.

Conclusion

This research demonstrates that unsupervised style transfer can effectively bridge the resolution gap in 3-D ultrasound. By moving away from "one-size-fits-all" AI, the switchable architecture provides a safety valve against GAN-induced hallucinations, allowing doctors to dial back enhancement if it appears unrealistic. This user-centric approach suggests a future where AI serves as a customizable, real-time filter on medical imaging consoles.

Summary updated

Abstract

In contrast to 2-D ultrasound (US) for uniaxial plane imaging, a 3-D US imaging system can visualize a volume along three axial planes. This allows for a full view of the anatomy, which is useful for gynecological (GYN) and obstetrical (OB) applications. Unfortunately, the 3-D US has an inherent limitation in resolution compared to the 2-D US. In the case of 3-D US with a 3-D mechanical probe, for example, the image quality is comparable along the beam direction, but significant deterioration in image quality is often observed in the other two axial image planes. To address this, here we propose a novel unsupervised deep learning approach to improve 3-D US image quality. In particular, using {\em unmatched} high-quality 2-D US images as a reference, we trained a recently proposed switchable CycleGAN architecture so that every mapping plane in 3-D US can learn the image quality of 2-D US images. Thanks to the switchable architecture, our network can also provide real-time control of image enhancement level based on user preference, which is ideal for a user-centric scanner setup. Extensive experiments with clinical evaluation confirm that our method offers significantly improved image quality as well user-friendly flexibility.
BibTeX
@article{huh2021tunable,
  author = {Jaeyoung Huh and Shujaat Khan and Sungjin Choi and Dongkuk Shin and Jeong Eun Lee and Eun Sun Lee and Jong Chul Ye},
  title = {{Tunable image quality control of 3-D ultrasound using switchable CycleGAN}},
  journal = {Medical Image Analysis},
  year = {2021},
  url = {https://openreview.net/forum?id=Aa2Ad5dVyx},
}
Figure