Tubular Shape Aware Data Generation for Semantic Segmentation in Medical Imaging

Ilyas Sirazitdinov, Heinrich Schulz, Axel Saalbach, Steffen Renisch, Dmitry V. DylovarXiv 2020 · DOI · OpenReview

TL;DR

This paper introduces a novel method for generating synthetic medical images of tubular objects (like catheters and wires) using Generative Adversarial Networks (GANs) regularized by a "tube-shape constraint." This approach significantly reduces the need for extensive manual annotation, enabling accurate segmentation models to be trained with only a handful of real, weakly-labeled images.

What Problem Does This Paper Address?

Accurately segmenting tubular structures like catheters, wires, and needles in medical images, particularly Chest X-rays (CXRs), is crucial for interventional radiology procedures. However, creating large, pixel-perfect annotated datasets for training deep learning models is extremely time-consuming and expensive. Existing methods often require paired image-mask data, which is scarce. This paper tackles the challenge of limited annotated medical data by proposing a way to generate realistic synthetic data that can effectively train segmentation models.

Methods

The core of the proposed method is a weakly-supervised synthetic data generation framework. It utilizes a Generative Adversarial Network (GAN) consisting of a generator (G) and a discriminator (D).

  • Generator (G): This is an encoder-decoder CNN that takes a clean CXR image and a synthetic tube mask as input. It then generates a synthetic tube painting that is added to the clean CXR to create a synthetic image with tubes.
  • Discriminator (D): This CNN is trained to distinguish between three classes: real CXRs with tubes, synthetic CXRs with tubes, and clean CXRs.
  • Tube-Shape Constraint (Frangi-based Regularization): This is the key innovation. To ensure the generated objects are indeed tubular, the generator is regularized using a differentiable Frangi filter. This filter analyzes the local image structure based on Hessian matrix eigenvalues, enforcing that generated tubular regions have specific eigenvalue properties characteristic of tubes. This constraint is crucial for producing realistic tube shapes and preventing the generator from creating arbitrary non-tubular objects.
  • Weakly-Supervised Training: The system is trained using only weakly-labeled data (10-20 images) and the prior knowledge of tube shapes, eliminating the need for pixel-wise ground truth masks for the synthetic data generation itself.

Datasets

  • Training Data for Synthetic Generation: The authors used a subset of the publicly available ChestX-ray14 dataset. They selected 1850 images, labeling them as either "clean" CXRs or CXRs containing tube-like objects. For synthetic data generation, they used clean CXRs and generated fake masks for tubular objects.
  • Evaluation Data: The segmentation models were evaluated on the ChestX-ray14 dataset, both with synthetic data and then fine-tuned on a small number of real, labeled images.

Key Insights and Ideas

  • Prior Shape Knowledge is Powerful: Explicitly incorporating prior knowledge about the desired shape (tubular) into the GAN training process is highly effective for generating realistic synthetic medical images. This is more effective than relying solely on adversarial training or simpler shape constraints.
  • Weak Supervision is Sufficient for Generation: The proposed method demonstrates that generating useful synthetic data for segmentation doesn't require pixel-perfect ground truth masks. Weakly-labeled data combined with shape priors can guide the GAN to produce high-quality synthetic images.
  • Synthetic Data as a Pre-training Booster: Synthetic data generated by this method acts as an excellent pre-training resource. Models trained on this synthetic data achieve significantly better performance when fine-tuned on even a small number of real, labeled images, outperforming models trained from scratch or on other types of synthetic data.
  • Frangi Filter's Adaptability: The Frangi filter, traditionally used for vessel enhancement, can be effectively adapted into a differentiable regularization term for GANs to enforce tubular structures in generated images.

Key Results

  • Improved Segmentation Performance: Models trained on the GAN-generated synthetic data and then fine-tuned on a small number of real labels achieved segmentation performance comparable to fully supervised models trained on significantly more real data.
  • Significant Dice Score Improvement: The Dice score improved from ~53-54% (using manual synthetic data) to ~65-66% after fine-tuning on a small set of real labels. Recall also saw a ~15-20% gain.
  • Efficiency: The method requires only a small number of weakly-labeled images (10-20) for fine-tuning to reach high accuracy, drastically reducing annotation effort.
  • Comparison to Baselines: The proposed GAN-based method with Frangi regularization outperformed baseline synthetic data generation methods ("smoothed mask" and "hand-drawn tubes") in terms of segmentation quality, as evidenced by higher Dice scores.

Limitations and Future Work

  • Domain Shift: The authors acknowledge a natural domain shift between synthetic and real CXRs. Synthetic images are primarily generated on "clean" CXRs, while real images can contain more pathologies and complex lung textures, leading to missed detections in these challenging areas.
  • Future Work: The paper suggests that further work on domain adaptation techniques could help reduce the gap between synthetic and real images. They also propose extending the tube-shape regularization to 3D for applications like CT scans. The method's applicability to other modalities beyond X-rays is also highlighted as a future direction.

Summary updated

Abstract

Chest X-ray is one of the most widespread examinations of the human body. In interventional radiology, its use is frequently associated with the need to visualize various tube-like objects, such as puncture needles, guiding sheaths, wires, and catheters. Detection and precise localization of these tube-like objects in the X-ray images is, therefore, of utmost value, catalyzing the development of accurate target-specific segmentation algorithms. Similar to the other medical imaging tasks, the manual pixel-wise annotation of the tubes is a resource-consuming process. In this work, we aim to alleviate the lack of the annotated images by using artificial data. Specifically, we present an approach for synthetic data generation of the tube-shaped objects, with a generative adversarial network being regularized with a prior-shape constraint. Our method eliminates the need for paired image--mask data and requires only a weakly-labeled dataset (10--20 images) to reach the accuracy of the fully-supervised models. We report the applicability of the approach for the task of segmenting tubes and catheters in the X-ray images, whereas the results should also hold for the other imaging modalities.
BibTeX
@misc{sirazitdinov2020tubular,
  author = {Ilyas Sirazitdinov and Heinrich Schulz and Axel Saalbach and Steffen Renisch and Dmitry V. Dylov},
  title = {{Tubular Shape Aware Data Generation for Semantic Segmentation in Medical Imaging}},
  year = {2020},
  eprint = {2010.00907},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.00907},
}
Figure