Dehazing Ultrasound Using Diffusion Models

Tristan S. W. Stevens, Faik C. Meral, Jason Yu, Iason Zacharias Apostolakis, Jean-Luc Robert, Ruud J. G. van SlounIEEE TMI 2024 · DOI

TL;DR

This paper introduces a novel method for removing "haze" artifacts in ultrasound images using diffusion models. By jointly modeling clean ultrasound signals and haze, their approach significantly improves image quality, making it easier to diagnose cardiac conditions, especially in challenging-to-image patients.

What Problem Does This Paper Address?

Cardiac ultrasound (echocardiography) is a vital tool for diagnosing heart disease, but its effectiveness is often hampered by poor image quality. A significant issue is "haze," an artifact caused by multipath reflections of ultrasound waves as they travel through the body. This haze can obscure crucial details, rendering images difficult or even impossible to interpret, particularly in patients with obesity or dense muscle structures. While harmonic imaging offers some improvement, it's not always sufficient. Existing denoising algorithms struggle with the structured and correlated nature of haze. This paper aims to address this by developing a method that can effectively remove haze while preserving subtle tissue details.

Methods

The core of this work lies in a novel dehazing framework that leverages diffusion models. These are powerful generative models capable of learning complex data distributions. The authors propose a joint posterior sampling framework that utilizes two separate diffusion models: one for the clean ultrasound signal (x) and another for the haze (h).

Key methodological contributions include:

  • Joint Conditional Diffusion Process: Instead of modeling clean signals and haze independently, they model them jointly, conditioned on the observed hazy ultrasound measurement (y). This is achieved by formulating a joint conditional reverse-time SDE.
  • RF Domain Processing: The entire dehazing process, including training the diffusion models, is performed in the radio-frequency (RF) domain. This allows them to exploit phase information and linearly separate signal and haze earlier in the processing chain, which is more advantageous than working with envelope-detected image data.
  • µ-Law Companding: To handle the high dynamic range of RF ultrasound data, which can hinder neural network training, they employ a µ-law companding technique. This is an invertible operation that compresses the dynamic range, making the data distribution more amenable to learning.
  • Patch-Based Inference: For computational efficiency and better generalization, the authors use a patch-based approach. They interleave the diffusion process across overlapping patches to ensure coherence and avoid artifacts.
  • Haze Estimation: The framework inherently estimates the haze, which can be independently acquired or generated using specialized phantoms, offering flexibility in training data acquisition.

Datasets

  • In-Vitro Data: A controlled experiment using a heart phantom and a dedicated haze phantom was used to collect RF data. This dataset was split into training (2x150 frames) and validation (2x38 frames) sets. Haze was artificially added to the clean data to create varying levels of haze for evaluation.
  • In-Vivo Data: Ultrasound data from five human volunteers was acquired using an X5-1C matrix phased array transducer. This dataset included various acoustic views (AP3/AP4, PLAX, PSAX) and consisted of 2640 frames from 6 volunteers. This data was split into:
  • Training set (1500 clean frames) for the tissue model.
  • Validation set (1020 frames) for validation.
  • A separate training set (120 frames) from a third volunteer, considered hazy, for augmentation.
  • A specialized haze-only training dataset (H) was generated using apodization methods.

Key Insights and Ideas

  • RF Domain Advantage: Performing dehazing in the RF domain, rather than the image domain, is crucial for preserving fine tissue details and exploiting phase information.
  • Diffusion Models for Structured Noise: Diffusion models are particularly well-suited for removing structured noise like haze, which traditional methods struggle with. Their ability to learn complex data distributions is key.
  • Joint Modeling is Powerful: Explicitly modeling both the clean signal and the haze simultaneously, and conditioning on the measurement, leads to more effective separation and reconstruction.
  • Companding for RF Data: The µ-law companding technique is essential for making RF ultrasound data trainable by diffusion models, addressing the challenge of its high dynamic range.
  • Patch-Based Inference for Scalability: The patch-based approach, with interleaving, allows for efficient processing of large ultrasound volumes without introducing stitching artifacts, making the method more practical.

Key Results

  • Improved Contrast and Detail Preservation: The proposed diffusion-based method significantly improves contrast in ultrasound images and, importantly, preserves low-level tissue signals that are often lost by baseline methods.
  • Quantitative Improvements:
    • In-Vitro: The method achieved improved PSNR scores across various haze levels compared to BM3D and NCSNv2.
  • In-Vivo: The method demonstrated improved generalized contrast-to-noise ratio (gCNR) scores across different acoustic windows and subjects, outperforming BM3D and a supervised NCSNv2 model.
  • Qualitative Improvements: Visual results show a noticeable reduction in haze, leading to clearer visualization of cardiac structures. The method also revealed low-level tissue details that were obscured in the original or baseline-processed images.
  • Downstream Task Performance: The dehazed images aided a pretrained EchoNet model in producing more accurate delineations of the left ventricle (Dice score of 0.891 vs. 0.880 for hazy input).

Limitations and Future Work

  • Computational Cost: The current diffusion dehazing method is not ready for real-time deployment due to the iterative nature of the sampling process and the need for two independent score networks, resulting in a high number of neural function evaluations.
  • Future Work:
    • Computational Efficiency: Exploring advancements in sampling schemes to enable real-time implementation.
  • Slow-Time Axis Utilization: Extending the method to utilize the slow-time axis (temporal information across frames) to further enhance haze characterization and potentially expedite inference.
  • Arbitrary Degradation Models: Investigating the substitution of the standard Gaussian corruption process with arbitrary degradation models, potentially incorporating inherent haze as destructive noise.
  • Broader Applicability: Assessing the method's performance across different ultrasound scanners, settings, and a more diverse patient cohort.
  • Adaptive Dehazing Control: Exploiting data consistency weighting terms for more fine-grained or adaptive control of the dehazing amount, potentially using depth-varying weights.

Summary updated

BibTeX
@article{stevens2024dehazing,
  author = {Tristan S. W. Stevens and Faik C. Meral and Jason Yu and Iason Zacharias Apostolakis and Jean-Luc Robert and Ruud J. G. van Sloun},
  title = {{Dehazing Ultrasound Using Diffusion Models}},
  journal = {IEEE Trans. Medical Imaging},
  year = {2024},
  doi = {10.1109/tmi.2024.3363460},
}
Figure