MimickNet, Matching Clinical Post-Processing Under Realistic Black-Box Constraints

Ouwen Huang, Will Long, Nick Bottenus, Gregg E. Trahey, Sina Farsiu, Mark L. Palmeri

Abstract

Image post-processing is used in clinical-grade ultrasound scanners to improve image quality (e.g., reduce speckle noise and enhance contrast). These post-processing techniques vary across manufacturers and are generally kept proprietary, which presents a challenge for researchers looking to match current clinical-grade workflows. We introduce a deep learning framework, MimickNet, that transforms raw conventional delay-and-summed (DAS) beams into the approximate post-processed images found on clinical-grade scanners. Training MimickNet only requires post-processed image samples from a scanner of interest without the need for explicit pairing to raw DAS data. This flexibility allows it to hypothetically approximate any manufacturer's post-processing without access to the pre-processed data. MimickNet generates images with an average similarity index measurement (SSIM) of 0.930±0.0892 on a 300 cineloop test set, and it generalizes to cardiac cineloops outside of our train-test distribution achieving an SSIM of 0.967±0.002. To our knowledge, this is the first work to establish deep learning models that closely approximate current clinical-grade ultrasound post-processing under realistic black-box constraints where before and after post-processing data is unavailable. MimickNet serves as a clinical post-processing baseline for future works in ultrasound image formation to compare against.

Introduction

In medical ultrasound imaging, a significant gap exists between academic research and clinical application. Researchers typically develop novel beamforming techniques—the process of converting raw acoustic echoes into visual data—using open-source systems. However, commercial clinical-grade scanners (e.g., Siemens, GE) apply proprietary post-processing layers to these raw beams to reduce speckle noise and enhance contrast. Because these algorithms are trade secrets, researchers often compare their raw results against other raw results, rather than the polished images clinicians actually use. MimickNet addresses this by providing an open-source framework to replicate these proprietary workflows.

Methods

The authors treat the replication of clinical post-processing as a style transfer task. The framework is designed to operate under two distinct constraints:

  1. Gray-Box (Paired): A scenario where both raw (pre-processed) and clinical (post-processed) images are available for the same frame. This establishes a performance ceiling using a standard U-Net architecture.
  2. Black-Box (Unpaired): The more realistic scenario where researchers have access to a set of raw images and a separate set of clinical images, but no direct pairing between them. For this, a CycleGAN framework is employed.
The MimickNet architecture features a U-Net style Generator and a convolutional Discriminator for adversarial training.

The core architecture, as shown in Figure 1, utilizes an encoder-decoder network with skip connections. The model is optimized using a combination of loss functions, including Mean Squared Error (MSE) and Structural Similarity Index (SSIM), to ensure the output reflects human perception of image quality.

Experimental Setup

The study utilized a robust dataset of 1,500 unique ultrasound cineloops, totaling over 39,000 frames. The targets included fetal, liver, and phantom (synthetic tissue) images collected across Siemens S2000, SC2000, and Verasonics Vantage scanners. To test the model's ability to generalize, the researchers evaluated it on cardiac cineloops, which were entirely excluded from the training distribution.

Results and Discussion

MimickNet demonstrated high fidelity in replicating clinical imagery. In the black-box test set, the model achieved an average SSIM of 0.930. Qualitatively, the model successfully learned to remove bright reflector artifacts and speckle noise, matching the "clean" appearance of clinical scanners.

Qualitative and quantitative results for liver and phantom images, including histograms showing the distribution of similarity ratios for contrast-structure and luminance.

As illustrated in Figure 2, the model excelled at matching contrast and structure. While it faced slight challenges in matching absolute luminance (brightness) under black-box constraints, this is considered a minor limitation as clinicians frequently adjust luminance via manual gain controls.

Key performance metrics include:

  • Generalization: The model achieved an SSIM of 0.967 on out-of-distribution cardiac data, suggesting it learned general ultrasound physics rather than specific anatomical features.
  • Efficiency: MimickNet is highly optimized, requiring 2000x fewer floating-point operations (FLOPS) than MobileNetV2. It achieves a real-time processing speed of 92 FPS on an NVIDIA P100 GPU.

Conclusion

MimickNet provides a critical baseline for ultrasound research, allowing for fair comparisons between new beamforming methods and industry-standard clinical outputs. By successfully approximating proprietary post-processing under black-box constraints, this framework enables researchers to validate the clinical relevance of their work without requiring access to manufacturer secrets. The software has been made open-source to facilitate community adoption.

Summary updated

Abstract

Image post-processing is used in clinical-grade ultrasound scanners to improve image quality (e.g., reduce speckle noise and enhance contrast). These post-processing techniques vary across manufacturers and are generally kept proprietary, which presents a challenge for researchers looking to match current clinical-grade workflows. We introduce a deep learning framework, MimickNet, that transforms raw conventional delay-and-summed (DAS) beams into the approximate post-processed images found on clinical-grade scanners. Training MimickNet only requires post-processed image samples from a scanner of interest without the need for explicit pairing to raw DAS data. This flexibility allows it to hypothetically approximate any manufacturer's post-processing without access to the pre-processed data. MimickNet generates images with an average similarity index measurement (SSIM) of 0.930$\pm$0.0892 on a 300 cineloop test set, and it generalizes to cardiac cineloops outside of our train-test distribution achieving an SSIM of 0.967$\pm$0.002. We also explore the theoretical SSIM achievable by evaluating MimickNet performance when trained under gray-box constraints (i.e., when both pre-processed and post-processed images are available). To our knowledge, this is the first work to establish deep learning models that closely approximate current clinical-grade ultrasound post-processing under realistic black-box constraints where before and after post-processing data is unavailable. MimickNet serves as a clinical post-processing baseline for future works in ultrasound image formation to compare against. To this end, we have made the MimickNet software open source.
BibTeX
@article{huang2020mimicknet,
  author = {Ouwen Huang and Will Long and Nick Bottenus and Gregg E. Trahey and Sina Farsiu and Mark L. Palmeri},
  title = {{MimickNet, Matching Clinical Post-Processing Under Realistic Black-Box Constraints}},
  journal = {IEEE Trans. Medical Imaging},
  year = {2020},
  url = {https://openreview.net/forum?id=OmxDf8CUMs},
}
Figure