Boosting Liver and Lesion Segmentation from CT Scans By Mask Mining

Karsten Roth, Jürgen Hesser, Tomasz K. Konopczynski

Abstract

We propose a novel procedure to improve liver and lesion segmentation from CT scans for U-Net based models. Our method extends standard segmentation pipelines to focus on higher target recall or reduction of noisy false-positive predictions, boosting overall segmentation performance. To achieve this, we include segmentation errors into a new learning process appended to the main training setup, allowing the model to find features which explain away previous errors. We evaluate this on semantically distinct architectures: cascaded two- and three-dimensional as well as combined learning setups for multitask segmentation. Liver and lesion segmentation data are provided by the Liver Tumor Segmentation challenge (LiTS), with an increase in dice score of up to 2 points.

Introduction

Accurate segmentation of the liver and its lesions from CT scans is critical for diagnosis and treatment planning. While U-Net architectures are the industry standard, squeezing extra performance out of them usually requires computationally expensive re-runs or complex hyperparameter tuning. Existing methods to handle segmentation errors often involve specialized loss functions or adversarial setups that limit which network architectures can be used. We present "Mask Mining," an architecture-independent "plug-and-play" pipeline that boosts performance by helping a network learn from its own specific failure modes.

Methodology: The Mask Mining Pipeline

The proposed pipeline consists of three main stages designed to "explain away" errors by retraining the model on its own mistakes.

  1. Initial Stage: A standard segmentation network (e.g., 2D or 3D U-Net) is trained to convergence on the original binary task.
  2. Mask Generation: The trained model performs a forward pass over the training data. Predictions are compared to ground truth to generate a four-class Finegrained Mined Segmentation Mask: True Negative (TN), False Positive (FP), False Negative (FN), and True Positive (TP).
  3. Final Stage (Finetuning): Four single-layer output channels are appended to the network as error prediction layers. The model is then retrained on these newly mined masks.
The Mask Mining Pipeline: Errors from the initial stage are used to create a 4-class mask for a secondary finetuning stage.

This secondary training converges rapidly due to weight initialization. By selecting specific loss functions during this stage—such as Pixel-Weighted Cross-Entropy (PWCE) or a sensitive Dice-based loss—users can retrospectively tune the model to prioritize reducing either false positives or false negatives.

Experimental Setup

The method was evaluated using the Liver Tumor Segmentation (LiTS) dataset, containing 131 3D abdominal CT scans with ground truth masks. Testing was conducted across three distinct architectures: Cascaded 2D, Cascaded 3D, and a Combined Cascaded 2D setup. An 85%/15% train/validation split was used, with final evaluation performed on 70 test volumes via the LiTS online submission portal.

Results and Discussion

The mask mining procedure provided a consistent performance boost across all tested setups. Quantitatively, the method increased the Dice score by up to 2 points. In the "Combined" (Cmb) setup, lesion segmentation on the online test set improved from a Dice score of 61.9 ± 0.2 to 63.0 ± 0.1.

Setup Online Test Dice (Liver) Online Test Dice (Lesion)
2D 95.3 ± 0.2 62.9 ± 0.3
2D + Inc. 95.5 ± 0.3 63.5 ± 0.2
Cmb 93.4 ± 0.3 61.9 ± 0.2
Cmb + Inc. 94.7 ± 0.3 63.0 ± 0.1

Qualitatively, the "Boosted Segmentation" significantly reduces false negatives (FN), particularly in small or difficult-to-detect lesions.

Qualitative results showing how "Boosted Segmentation" (right) significantly reduces false negatives (red) compared to the original model.

The mining step forces the network to find features that specifically characterize its previous mistakes, helping it distinguish between "hard" background pixels and actual lesion tissue. This retrospective control allows for performance tweaking without re-architecting the base model.

Conclusion

Mask mining offers a robust, architecture-independent method for improving medical image segmentation. By explicitly modeling errors, we achieve higher sensitivity and reduced noise. Future work will explore iterative mask mining to further enhance sensitivity, potentially offering a general-purpose tool for critical clinical tasks where missing a tumor is of paramount concern.

Summary updated

Abstract

We propose a novel procedure to improve liver and lesion segmentation from CT scans for U-Net based models. Our method extends standard segmentation pipelines to focus on higher target recall or reduction of noisy false-positive predictions, boosting overall segmentation performance. To achieve this, we include segmentation errors into a new learning process appended to the main training setup, allowing the model to find features which explain away previous errors. We evaluate this on semantically distinct architectures: cascaded two- and three-dimensional as well as combined learning setups for multitask segmentation. Liver and lesion segmentation data are provided by the Liver Tumor Segmentation challenge (LiTS), with an increase in dice score of up to 2 points.
BibTeX
@inproceedings{roth2020boosting,
  author = {Karsten Roth and Jürgen Hesser and Tomasz K. Konopczynski},
  title = {{Boosting Liver and Lesion Segmentation from CT Scans By Mask Mining}},
  booktitle = {ISBI},
  year = {2020},
  url = {https://openreview.net/forum?id=_LRRzYwrVTi},
}

Loading related content...

Figure