MedUHIP: Towards Human-In-the-Loop Medical Segmentation

Jiayuan Zhu, Junde WuarXiv 2024 · OpenReview

TL;DR

This paper introduces MedUHIP, a novel human-in-the-loop system that tackles the inherent uncertainty in medical image segmentation by combining an uncertainty-aware model with iterative clinician feedback. MedUHIP generates multiple plausible segmentations and learns individual clinician preferences to refine them, leading to more accurate and reliable segmentations for clinical use.

What Problem Does This Paper Address?

Medical image segmentation is crucial for diagnosis and treatment, but it's plagued by inherent uncertainties. Unlike natural images with clear boundaries, medical images often have ambiguous tissue borders. This ambiguity leads to variations in annotations even among experienced clinicians, creating "uncertain ground truth." This inconsistency can hinder accurate clinical interpretation and impact subsequent medical interventions. Existing automated segmentation methods, often adapted from natural image processing, struggle to account for this inherent uncertainty. While some approaches generate multiple segmentations to represent uncertainty, they can be overwhelming for clinicians to review. Human-in-the-loop methods aim to address this by incorporating clinician feedback, but they often require extensive interaction or struggle to capture individual preferences effectively.

Methods

MedUHIP proposes a novel approach that integrates two key components:

  • Uncertainty-Aware Model: This component generates a set of plausible segmentations to capture the inherent ambiguity in medical images. It does this by sampling from a "sampling space" that represents potential variations.
  • Human-in-the-Loop Interaction: This is where the system becomes truly interactive. Clinicians provide feedback (e.g., clicks, scribbles) on the generated segmentations. The system uses this feedback to refine its understanding of the clinician's preferences.

The core of the interaction lies in the Sampling Net. This module learns to adapt the sampling space based on clinician input. When a clinician is unsatisfied with the initial soft prediction (an aggregation of the plausible segmentations), the system highlights candidate regions for improvement. The Sampling Net then modifies the sampling space to generate segmentations that better align with the clinician's specific preferences in the next iteration. This iterative process allows the model to learn individual clinician behaviors and biases, leading to a final segmentation that is not only algorithmically precise but also clinically validated.

Datasets

The authors evaluated MedUHIP on several publicly available multi-clinician annotated datasets:

  • REFUGE2: A fundus image dataset for glaucoma analysis, featuring optic-cup segmentation. It's annotated by seven ophthalmologists.
  • LIDC-IDRI: A dataset of 3D lung CT scans with annotations for lung abnormalities. The authors used a pre-processed version with 2D CT images.
  • QUBIQ: A dataset designed to investigate inter-clinician variability in medical image segmentation. It includes MRI and CT scans for various tasks like brain tumor, prostate, and kidney segmentation, with multiple annotations per case.

Key Insights and Ideas

  • Clinician-Specific Preferences Matter: The paper strongly suggests that individual clinicians have consistent interaction behaviors, but these behaviors differ across clinicians. MedUHIP leverages this by learning these unique preferences.
  • Uncertainty is Not Just Noise: The paper frames uncertainty not as something to be eliminated, but as an inherent characteristic of medical images that needs to be modeled and managed.
  • Synergy of Automation and Expertise: MedUHIP demonstrates that combining automated segmentation with targeted clinician feedback can lead to superior results compared to purely automated or less interactive methods.
  • Efficiency Through Adaptive Sampling: The Sampling Net's ability to adapt the sampling space based on clinician interaction is key to achieving good results with fewer interactions.

Key Results

  • Superior Segmentation Performance: MedUHIP consistently outperformed deterministic, uncertainty-aware, and other interactive models across multiple datasets, achieving an average Dice Score of 89.68% after three interactions.
  • Significant Improvement on LIDC: The LIDC segmentation task saw a notable improvement of ~20% over existing SOTA methods.
  • Fewer Interactions Required: MedUHIP demonstrated superior efficiency, requiring fewer interactions on average to reach high-performance segmentation results compared to other interactive methods like SAM and MedSAM. For example, it achieved 80% Dice Score on REFUGE2 in 2.72 interactions on average, compared to SAM's 3.65.
  • Robustness Across Interaction Strategies: The method showed strong performance regardless of the clinician's interaction strategy (e.g., random selection, focusing on disagreements, or self-correction).

Limitations and Future Work

  • Limitations: The paper acknowledges that while MedUHIP is competitive, it doesn't always achieve the absolute fewest clicks for every task. The performance is also dependent on the quality and consistency of clinician input.
  • Future Work: The authors plan to release their code to encourage further research. The success of MedUHIP suggests potential for broader applications in medical image analysis and other fields where human expertise is crucial for refining automated systems. The exploration of different interaction modalities and more sophisticated ways to model clinician preferences could be future directions.

Summary updated

Abstract

Although segmenting natural images has shown impressive performance, these techniques cannot be directly applied to medical image segmentation. Medical image segmentation is particularly complicated by inherent uncertainties. For instance, the ambiguous boundaries of tissues can lead to diverse but plausible annotations from different clinicians. These uncertainties cause significant discrepancies in clinical interpretations and impact subsequent medical interventions. Therefore, achieving quantitative segmentations from uncertain medical images becomes crucial in clinical practice. To address this, we propose a novel approach that integrates an \textbf{uncertainty-aware model} with \textbf{human-in-the-loop interaction}. The uncertainty-aware model proposes several plausible segmentations to address the uncertainties inherent in medical images, while the human-in-the-loop interaction iteratively modifies the segmentation under clinician supervision. This collaborative model ensures that segmentation is not solely dependent on automated techniques but is also refined through clinician expertise. As a result, our approach represents a significant advancement in the field which enhances the safety of medical image segmentation. It not only offers a comprehensive solution to produce quantitative segmentation from inherent uncertain medical images, but also establishes a synergistic balance between algorithmic precision and clincian knowledge. We evaluated our method on various publicly available multi-clinician annotated datasets: REFUGE2, LIDC-IDRI and QUBIQ. Our method showcases superior segmentation capabilities, outperforming a wide range of deterministic and uncertainty-aware models. We also demonstrated that our model produced significantly better results with fewer interactions compared to previous interactive models. We will release the code to foster further research in this area.
BibTeX
@misc{zhu2024meduhip,
  author = {Jiayuan Zhu and Junde Wu},
  title = {{MedUHIP: Towards Human-In-the-Loop Medical Segmentation}},
  year = {2024},
  eprint = {2408.01620},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.01620},
}
Figure