TSE-GAN: strain elastography using generative adversarial network for thyroid disease diagnosis

Anping Song, Tianyi Li, Xuehai Ding, Mingye Wu, Ren WangFrontiers in Bioengineering and Biotechnology 2024 · DOI

TL;DR

TSE-GAN is a novel deep learning framework that automatically generates thyroid strain elastograms from standard B-mode ultrasound images. By utilizing a global-to-local architecture and a perceptually-driven Lab color space loss, the model eliminates the need for inconsistent manual tissue compression while outperforming existing image-to-image translation methods in clinical diagnostic accuracy.

What Problem Does This Paper Address?

Strain elastography (SE) is a powerful tool for detecting thyroid cancer because malignant nodules are typically stiffer than healthy tissue. However, traditional SE has a major "human element" problem: it requires a sonographer to manually compress the tissue with a transducer to measure displacement. This process is difficult to standardize, highly subjective, and relies on oversimplified physical assumptions (like tissue being perfectly linear or elastic).

The authors aim to bypass these physical limitations by treating elastogram generation as an image-to-image translation task. The goal is to take a standard, non-invasive B-mode ultrasound image and "hallucinate" a medically accurate strain elastogram that a doctor can use for diagnosis.

Methods

The researchers proposed TSE-GAN, a conditional Generative Adversarial Network (cGAN) with several specialized architectural choices:

  • Global-to-Local Generator: The generator is split into three components. A global generator handles the overall image structure, a local generator focuses specifically on the thyroid nodule (using a pre-trained segmentation network to find it), and a content revisor uses channel attention to refine the foreground and background details.
  • Deformable Convolutions (DefC): Unlike standard rigid convolutional kernels, DefC can adapt its shape to the non-rigid, irregular deformations of soft thyroid tissue. This allows the network to better model how different parts of the tissue "stretch" or "squish" during a simulated compression.
  • Lab-based Color Loss: Most GANs operate in RGB space. However, the authors observed that thyroid elastograms (which use color to represent stiffness) have a much more concentrated probability distribution in the CIE Lab color space. By calculating loss in Lab space, the model generates colors that are more perceptually accurate and clinically meaningful.
  • Dual Discriminators: The system uses two PatchGAN discriminators to evaluate the generated images at both a global scale and a local (nodule-specific) scale, ensuring the "fake" elastograms look realistic at every level of detail.

Datasets

The study utilized a proprietary clinical dataset provided by Shanghai Sixth People’s Hospital.

  • Composition: 1,224 pairs of B-mode ultrasound and corresponding strain elastography samples.
  • Source: Data was collected from 745 patients between 2019 and 2022 using Esaote MylabTwice ultrasound units.
  • Preprocessing: The dataset was cleaned to exclude nodules with heavy calcification or cystic characteristics, resulting in 1,129 training pairs and 95 testing pairs.

Key Insights and Ideas

  • The "Lab" Advantage: One of the most interesting "aha" moments is the shift from RGB to Lab color space. Because Lab space is designed to be perceptually uniform (matching how humans see color), the loss function more effectively penalizes the subtle color shifts that would lead a doctor to misjudge tissue stiffness.
  • Context Matters: By using a pre-trained segmentation model (BTNet) to guide the local generator, the model "knows" where the nodule is. This prevents the common GAN issue where the model generates a high-quality image but places the most important features in the wrong spot.
  • Modeling Non-Rigidity: The inclusion of deformable convolutions acknowledges that medical imaging isn't just about pixels; it’s about underlying biomechanics. Standard CNNs struggle with the fluid shapes of anatomy, but deformable kernels provide the flexibility needed for "virtual" elastography.

Key Results

TSE-GAN was compared against seven state-of-the-art models (including Pix2pixHD, CycleGAN, and AttentionGAN).

  • Quantitative Performance: TSE-GAN achieved the highest scores across the board: PSNR of 28.535 and SSIM of 0.413.
  • Clinical Accuracy: When evaluated using the Rago criterion (a 1–5 scale for nodule stiffness), TSE-GAN achieved a mean accuracy of 74.74%, significantly higher than the next best model, Pix2pixHD (57.89%).
  • Specialist Perceptual Study: In a "Turing test" for ultrasound, three professional sonographers were asked to distinguish real images from fake ones. TSE-GAN achieved a mean accuracy of 76.3%, meaning the generated images were realistic enough to frequently confuse experts.

Limitations and Future Work

Despite the strong results, the authors note a few hurdles:

  • Anatomical Challenges: The model currently struggles with the "greater tubercles" (specific anatomical landmarks), where translation accuracy drops.
  • Ambiguous Cases: Differentiating between Score 2 and Score 3 (mildly stiff vs. moderately stiff) remains difficult because the color distributions in these categories overlap significantly.
  • Segmentation Dependency: The local generator's success depends heavily on the accuracy of the initial segmentation. If the pre-trained model misses the nodule, the local generator cannot correct it.
  • Future Direction: The authors plan to conduct more in-depth research on nodule feature extraction to improve the quantification of the elasticity scoring process.

Summary updated

Abstract

Over the past 35 years, studies conducted worldwide have revealed a threefold increase in the incidence of thyroid cancer. Strain elastography is a new imaging technique to identify benign and malignant thyroid nodules due to its sensitivity to tissue stiffness. However, there are certain limitations of this technique, particularly in terms of standardization of the compression process, evaluation of results and several assumptions used in commercial strain elastography modes for the purpose of simplifying imaging analysis. In this work, we propose a novel conditional generative adversarial network (TSE-GAN) for automatically generating thyroid strain elastograms, which adopts a global-to-local architecture to improve the ability of extracting multi-scale features and develops an adaptive deformable U-net structure in the sub-generator to apply effective deformation. Furthermore, we introduce a Lab-based loss function to induce the networks to generate realistic thyroid elastograms that conform to the probability distribution of the target domain. Qualitative and quantitative assessments are conducted on a clinical dataset provided by Shanghai Sixth People's Hospital. Experimental results demonstrate that thyroid elastograms generated by the proposed TSE-GAN outperform state-of-the-art image translation methods in meeting the needs of clinical diagnostic applications and providing practical value.
BibTeX
@inproceedings{song2024tse,
  author = {Anping Song and Tianyi Li and Xuehai Ding and Mingye Wu and Ren Wang},
  title = {{TSE-GAN: strain elastography using generative adversarial network for thyroid disease diagnosis}},
  booktitle = {Frontiers in Bioengineering and Biotechnology},
  year = {2024},
  doi = {10.3389/fbioe.2024.1330713},
}
Figure