Hands-Free Segmentation of Medical Volumes via Binary Inputs

Florian Dubost, Loïc Peter, Christian Rupprecht, Benjamín Gutiérrez-Becker, Nassir Navab

TL;DR

This paper introduces a "hands-free" interactive method for segmenting 3D medical volumes by having users answer simple "Yes/No" questions about whether a specific voxel belongs to an organ. By combining a learned shape prior with a clever adaptive sampling strategy, the system can refine complex 3D segmentations in real-time using only binary inputs, such as a foot pedal or voice commands.

What Problem Does This Paper Address?

Interactive segmentation is a powerful tool in medical imaging, allowing clinicians to refine algorithmic outputs for surgery planning or prosthesis design. However, traditional methods usually require "hands-on" interaction—drawing scribbles or bounding boxes.

In a sterile surgical environment, this is problematic: clinicians' hands are occupied, and navigating through 3D slices on a screen is time-consuming and overwhelming. The authors aim to bridge this gap by creating a system that requires minimal cognitive and physical load, allowing a user to guide a segmentation algorithm through simple binary feedback.

Methods

The core of the approach is a "Twenty Questions" style interaction. The system presents a voxel to the user and asks: "Is this voxel inside the object to segment?"

To make this efficient, the authors employ several key components:

  • Probability Map: An initial guess of the organ's location is generated offline using an AdaBoost classifier trained on Haar features.
  • Relaxed Shape Prior: Instead of pixel-perfect accuracy, the system uses a statistical shape model (based on PCA of signed distance functions). This allows the algorithm to represent 3D shapes using only a few parameters (translation, rotation, size, and shape eigenmodes), making real-time computation feasible.
  • MCMC Sampling: The algorithm uses the Metropolis-Hastings algorithm to sample a set of "candidate" segmentations that are consistent with both the initial probability map and the user's previous answers.
  • Dichotomic Search: To choose the next question, the system identifies the voxel where the candidate segmentations disagree the most. Answering this question effectively "halves" the space of possible segmentations.
  • Adaptive Weighting ($\beta$): This is a novel contribution. The system dynamically adjusts how much it trusts the initial probability map versus the user's manual "seeds." If the initial map is misleading, the algorithm detects the conflict and increases the weight of the user's input.
Fig. 1. Diagram summarizing the processes of our method. The user (1) com- municates for instance via a pedal (2) with the algorithm (3) which outputs question voxels to the user (4). Using a probability map learnt offline (Sec. 2.1) and previous answers from the user, the algorithm samples several segmentations (Sec. 2.2) and finds the area where they disagree the most. The question voxel is taken within this area (Sec. 2.3). The question is of the form “Is this voxel inside the object to segment?”. The answer of the user provides a seed which halves the space of candidates segmentations. The final segmentation is computed from the set of seeds provided by the user and by running a last segmentation sampling procedure (Sec. 2.4).

Datasets

The method was evaluated using the Prostate Segmentation Challenge MICCAI09 dataset. This consists of 15 3D T2-weighted MRI volumes. The images have a resolution of $0.55 \times 0.55 \times 5\text{mm}^3$ and an average size of $256 \times 256 \times 32$ voxels.

Key Insights and Ideas

  • Binary Inputs are Sufficient: The paper proves that you don't need complex mouse movements to guide 3D segmentation; a sequence of well-chosen binary points (seeds) can effectively constrain a shape model.
  • Learning to Ignore the Prior: The most interesting technical detail is the adaptive weighting parameter $\beta$. In many interactive systems, a bad initial model can "trap" the results. Here, the system uses online transfer learning principles to realize when the user's answers contradict the probability map, allowing it to pivot away from a poor initial guess.
  • Efficiency through Shape Models: By operating in the parameter space of a shape model rather than the voxel space, the MCMC sampling becomes fast enough for interactive use (averaging ~4 seconds between questions).
Illustration of the algorithm selecting a question voxel (green dot) in an area of high uncertainty among candidate segmentations.

Key Results

  • Performance: The method outperformed two baselines: a simple thresholded probability map and a version of the algorithm that asked questions at random voxels.
  • Speed: The average computation time between questions was 4.2 seconds, which is low enough for a clinician to use interactively during a procedure.
  • Adaptability: In experiments with "synthetic" misleading probability maps (where the initial guess was completely wrong), the adaptive $\beta$ parameter allowed the system to ignore the bad prior and still achieve a high Dice Similarity Coefficient (DSC) based on user input.
  • Accuracy: On the prostate dataset, the method achieved a DSC of approximately 0.80. The authors noted that this score was largely capped by the limited "freedom" of the chosen shape model rather than the interaction method itself.

Limitations and Future Work

The primary limitation identified is the shape model's rigidity. Because the system relies on a pre-trained statistical shape model, it struggles to segment "unusual" or highly pathological organs that fall outside the training distribution (e.g., Patient 73 in their study).

Future work suggested by the authors includes:

  • Updating the probability map itself in real-time based on user answers.
  • Combining the current framework with unsupervised models to better handle novel shapes.
  • Exploring more sophisticated shape models to increase the upper bound of segmentation accuracy.

Summary updated

Abstract

We propose a novel hands-free method to interactively segment 3D medical volumes. In our scenario, a human user progressively segments an organ by answering a series of questions of the form "Is this voxel inside the object to segment?". At each iteration, the chosen question is defined as the one halving a set of candidate segmentations given the answered questions. For a quick and efficient exploration, these segmentations are sampled according to the Metropolis-Hastings algorithm. Our sampling technique relies on a combination of relaxed shape prior, learnt probability map and consistency with previous answers. We demonstrate the potential of our strategy on a prostate segmentation MRI dataset. Through the study of failure cases with synthetic examples, we demonstrate the adaptation potential of our method. We also show that our method outperforms two intuitive baselines: one based on random questions, the other one being the thresholded probability map.
BibTeX
@misc{dubost2016hands,
  author = {Florian Dubost and Loïc Peter and Christian Rupprecht and Benjamín Gutiérrez-Becker and Nassir Navab},
  title = {{Hands-Free Segmentation of Medical Volumes via Binary Inputs}},
  year = {2016},
  eprint = {1609.06192},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1609.06192},
}
Figure