An Unsupervised Approach to Ultrasound Elastography with End-to-end Strain Regularisation

Rémi Delaunay, Yipeng Hu, Tom VercauterenMICCAI 2020 · OpenReview

TL;DR

This paper introduces a novel deep learning approach for ultrasound elastography that can be trained without ground truth data, making it more adaptable to real-world clinical scenarios. By optimizing a similarity metric between ultrasound frames and incorporating a strain smoothness regularization, the method achieves accurate displacement and strain estimation, paving the way for more efficient and accessible ultrasound-based tissue characterization.

What Problem Does This Paper Address?

Ultrasound elastography (USE) is a valuable imaging technique for assessing tissue stiffness, crucial for diagnosing conditions like liver disease or tumors. A core challenge in USE is accurately estimating the tiny displacements (strain) within tissues caused by external compression. This estimation, known as time-delay estimation, is computationally intensive and sensitive to noise and decorrelation (changes in tissue appearance between frames). Existing methods often struggle with real-time performance or require precise ground truth data for training, which is difficult to obtain in clinical settings. This paper aims to overcome these limitations by developing an efficient, unsupervised deep learning method for strain estimation.

Methods

The authors propose a convolutional neural network (CNN) that takes a pair of pre- and post-compression ultrasound radio-frequency (RF) frames as input and outputs a dense displacement field (DDF). The key innovation lies in the unsupervised training approach. Instead of relying on ground truth displacement maps, the network is trained by minimizing a similarity metric (specifically, negative local normalized cross-correlation or NCC) between the reference (pre-compression) image and the warped compressed image.

Furthermore, they introduce a novel regularization term that directly optimizes the smoothness of the strain prediction. This is achieved by penalizing the second spatial derivatives of the predicted displacement field, which helps to preserve displacement continuity and produce more accurate strain maps. The overall optimization problem balances the similarity between images with this strain smoothness constraint. The network architecture itself is based on a standard encoder-decoder structure with skip connections, adapted from existing image registration frameworks.

Datasets

The method was evaluated on two types of datasets:

  • Simulated Ultrasound Data: Generated using the Field-II software, these datasets included digital phantoms with cylindrical inclusions of varying sizes and positions, mimicking typical ultrasound speckle patterns. The simulations incorporated variations in Young's modulus, axial compression, and added white Gaussian noise to enhance robustness. A total of 160 image pairs were simulated, with 100 for training, 30 for validation, and 30 for testing.
  • In Vivo Data: RF data was collected from the arms of three healthy human volunteers using a Cicada 128PX system with a 10 MHz linear probe. This dataset consisted of 1300 image pairs for training and 300 pairs for validation and testing. The in vivo data captured real-world complexities, including potential blood vessels.

Key Insights and Ideas

  • Unsupervised Learning for Elastography: The ability to train a deep learning model for elastography without ground truth labels is a significant breakthrough. This bypasses the need for complex and often unavailable ground truth data, making the method directly applicable to clinical ultrasound data.
  • End-to-End Strain Regularization: Directly optimizing for strain smoothness, rather than just displacement continuity, leads to improved contrast and quality in the resulting strain images. This suggests a more direct pathway to generating clinically relevant elastograms.
  • Bridging Image Registration and Elastography: The paper effectively leverages techniques from image registration (e.g., DDF prediction, similarity metrics) and adapts them for the specific challenges of ultrasound elastography, demonstrating the transferability of deep learning methods across related imaging tasks.

Key Results

  • Simulation Performance: The unsupervised method achieved an average Contrast-to-Noise Ratio (CNR) of 7.70 and a Signal-to-Noise Ratio (SNR) of 6.95 on the simulated dataset. This was comparable to or better than the supervised network (CNR: 4.22, SNR: 7.79) and competitive with the state-of-the-art OVERWIND method (CNR: 9.31, SNR: 6.33).
  • In Vivo Performance: For the in vivo data, the unsupervised method achieved an average CNR of 7 and an average SNR of 0.31. While the SNR is lower than OVERWIND (0.7), the CNR is comparable (8.43). Local Normalized Cross-Correlation (LNCC) scores, indicating displacement field accuracy, were high for both methods (e.g., 0.87 for unsupervised vs. 0.90 for OVERWIND in one case).
  • Real-time Capability: The method demonstrated a strain prediction rate of approximately 13 frames per second on a 12GB NVIDIA GTX-1080ti GPU, highlighting its potential for real-time applications.

Limitations and Future Work

  • SNR in In Vivo Data: The paper notes a lower SNR for the unsupervised method on in vivo data compared to OVERWIND. This could be an area for future improvement.
  • Parameter Tuning: While the method is largely unsupervised, the regularization weight ($\alpha$) still requires tuning. Future work could explore more automated or adaptive methods for setting this parameter.
  • Generalizability: While validated on simulations and healthy volunteers, further testing on pathological tissues and a wider range of clinical scenarios would be beneficial to fully assess its robustness and clinical utility.
  • Real-time Performance Comparison: The paper mentions OVERWIND's real-time performance was not quantitatively reported, suggesting a need for direct, comprehensive real-time comparisons.

Summary updated

Abstract

Quasi-static ultrasound elastography (USE) is an imaging modality that consists of determining a measure of deformation (i.e. strain) of soft tissue in response to an applied mechanical force. The strain is generally determined by estimating the displacement between successive ultrasound frames acquired before and after applying manual compression. The computational efficiency and accuracy of the displacement prediction, also known as time-delay estimation, are key challenges for real-time USE applications. In this paper, we present a novel deep-learning method for efficient time-delay estimation between ultrasound radio-frequency (RF) data. The proposed method consists of a convolutional neural network (CNN) that predicts a displacement field between a pair of pre- and post-compression ultrasound RF frames. The network is trained in an unsupervised way, by optimizing a similarity metric between the reference and compressed image. We also introduce a new regularization term that preserves displacement continuity by directly optimizing the strain smoothness. We validated the performance of our method by using both ultrasound simulation and in vivo data on healthy volunteers. We also compared the performance of our method with a state-of-the-art method called OVERWIND [17]. Average contrast-to-noise ratio (CNR) and signal-to-noise ratio (SNR) of our method in 30 simulation and 3 in vivo image pairs are 7.70 and 6.95, 7 and 0.31, respectively. Our results suggest that our approach can effectively predict accurate strain images. The unsupervised aspect of our approach represents a great potential for the use of deep learning application for the analysis of clinical ultrasound data.
BibTeX
@inproceedings{delaunay2020unsupervised,
  author = {Rémi Delaunay and Yipeng Hu and Tom Vercauteren},
  title = {{An Unsupervised Approach to Ultrasound Elastography with End-to-end Strain Regularisation}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  url = {https://openreview.net/forum?id=88J_RvO3W5R},
}
Figure