Mask Mining for Improved Liver Lesion Segmentation

Karsten Roth, Jürgen Hesser, Tomasz K. Konopczynski

TL;DR

Researchers have developed a "mask mining" post-training procedure that boosts the performance of U-Net models in liver and lesion segmentation. By explicitly retraining a model on its own mistakes—categorized as false positives and false negatives—the method "explains away" errors and allows researchers to retrospectively tune for higher recall or precision without changing the underlying architecture.

What Problem Does This Paper Address?

Accurate segmentation of the liver and its lesions from CT scans is critical for diagnosis and treatment planning. While U-Net architectures are the industry standard, squeezing extra performance out of them usually requires computationally expensive re-runs or complex hyperparameter tuning.

Existing methods to handle segmentation errors often involve specialized loss functions (like Tversky loss) that require heavy tuning or adversarial setups that limit which network architectures can be used. The authors sought a simpler, architecture-independent "plug-and-play" pipeline that could reliably boost performance by helping a network learn from its own specific failure modes.

Methods

The proposed "Mask Mining" pipeline consists of three main stages:

  1. Initial Stage: A standard segmentation network (like a 2D or 3D U-Net) is trained to convergence on the original binary task (e.g., lesion vs. background).
  2. Mask Generation: The trained model performs a single forward pass over the training data. Its predictions are compared to the ground truth to generate a new, four-class Finegrained Mined Segmentation Mask: True Negative (TN), False Positive (FP), False Negative (FN), and True Positive (TP).
  3. Final Stage (Finetuning): Four single-layer output channels are appended to the network to serve as error prediction layers. The model is then retrained on these newly mined masks.
The Mask Mining Pipeline: Errors from the initial stage are used to create a 4-class mask for a secondary finetuning stage.

The authors highlight that this secondary training converges much faster because the weights are already initialized. Furthermore, by choosing different loss functions during this stage—such as Pixel-Weighted Cross-Entropy (PWCE) or a sensitive Dice-based loss—users can steer the model to prioritize reducing either false positives or false negatives.

Datasets

The method was evaluated using the Liver Tumor Segmentation (LiTS) dataset, which contains:

  • 131 3D abdominal CT scans with ground truth masks for both the liver and lesions.
  • 70 test volumes evaluated via an online submission portal.

The researchers used an 85%/15% train/validation split and tested the method across three distinct architectures: Cascaded 2D, Cascaded 3D, and a Combined Cascaded 2D setup.

Key Insights and Ideas

  • "Explaining Away" Mistakes: Instead of just penalizing errors, the mining step forces the network to find features that specifically characterize its previous mistakes. This helps the model distinguish between "hard" background pixels (previously FPs) and actual lesion tissue.
  • Retrospective Control: One of the most powerful aspects of this method is the ability to tweak performance after the initial training. As shown in Figure 4, using a Dice-based loss in the mining step significantly boosts recall (reducing red FN areas), while a PWCE-based loss can be used to increase robustness against noise.
  • Architecture Independence: Because the mining happens as a post-training step on the output layers, it works regardless of whether the base model is 2D, 3D, or a complex multi-task cascade.
Qualitative results showing how "Boosted Segmentation" (right) significantly reduces false negatives (red) compared to the original model.

Key Results

The mask mining procedure provided a consistent performance boost across all tested setups:

  • Dice Score Improvement: The method increased the Dice score by up to 2 points.
  • Stability: Error inclusion reduced "seed-dependent variation," meaning the results were more consistent across different training runs.
  • Quantitative Gains: In the "Combined" (Cmb) setup, lesion segmentation on the online test set improved from a Dice score of 61.9 ± 0.2 to 63.0 ± 0.1.
Setup Online Test Dice (Liver) Online Test Dice (Lesion)
2D 95.3 ± 0.2 62.9 ± 0.3
2D + Inc. 95.5 ± 0.3 63.5 ± 0.2
Cmb 93.4 ± 0.3 61.9 ± 0.2
Cmb + Inc. 94.7 ± 0.3 63.0 ± 0.1

Limitations and Future Work

While the method provides a clear boost, the authors note that they still observe limitations in the absolute Dice score, likely due to the inherent difficulty of the LiTS dataset and the capacity of the base U-Net models.

The authors suggest that iterative mask mining—repeating the error-generation and finetuning steps multiple times—could potentially lead to even higher sensitivity. They also see this as a promising general-purpose tool for other medical imaging tasks where reducing false negatives (missing a tumor) is of paramount clinical importance.

Summary updated

Abstract

We propose a novel procedure to improve liver and lesion segmentation from CT scans for U-Net based models. Our method extends standard segmentation pipelines to focus on higher target recall or reduction of noisy false-positive predictions, boosting overall segmentation performance. To achieve this, we include segmentation errors into a new learning process appended to the main training setup, allowing the model to find features which explain away previous errors. We evaluate this on semantically distinct architectures: cascaded two-and three-dimensional as well as combined learning setups for multitask segmentation. Liver and lesion segmentation data are provided by the Liver Tumor Segmentation challenge (LiTS), with an increase in dice score of up to 2 points.
BibTeX
@inproceedings{roth2020mask,
  author = {Karsten Roth and Jürgen Hesser and Tomasz K. Konopczynski},
  title = {{Mask Mining for Improved Liver Lesion Segmentation}},
  booktitle = {ISBI},
  year = {2020},
  url = {https://openreview.net/forum?id=_LRRzYwrVTi},
}

Loading related content...

Figure