<title>Shape-based models for interactive segmentation of medical images</title>

Kevin P. Hinshaw, Russ B. Altman, James F. BrinkleyProceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE 1995 · DOI

TL;DR

This paper introduces "Scanner," an interactive system that uses shape models to assist in medical image segmentation, significantly reducing manual effort. They compare two shape models, a "min/max" and a "probabilistic" model, finding that both improve segmentation efficiency, with the probabilistic model showing promise but facing computational challenges.

What Problem Does This Paper Address?

Accurate segmentation of medical images is crucial for applications like radiation treatment planning, where clinicians must manually trace critical structures on numerous images. This process is time-consuming and prone to error, especially when adjacent soft tissues have similar densities, making traditional edge detection unreliable. The paper addresses the need for a more automated and efficient segmentation approach by leveraging the inherent shape similarities of anatomical structures across different images and patients.

Methods

The core of the paper is the "Scanner" system, which implements two types of shape models:

  • Min/max model: This model represents shape using uncertainty intervals for radial lengths from a contour's center. It also incorporates a matrix of "shape constraints" that define acceptable ratios between radial lengths, ensuring that the overall contour maintains a consistent shape. Constraint propagation is used to tighten these intervals as more information becomes available.
  • Probabilistic model: This model uses probability distributions (mean and variance) for each radial's length. It also captures covariation between radial lengths. This approach allows for more flexible updates to radial lengths, as it doesn't fix a value once assigned. The Kalman filter is used to update these distributions as edge data is incorporated.

Both models are integrated into the "Scanner" system, which provides an interactive interface for users to guide the segmentation process. Users define a contour's initial orientation and scale, and the system then uses the shape model to guide an edge detector. Users can also manually correct segmentation errors.

Datasets

The paper describes an evaluation using fifteen biological shape classes, representing cross-sections of organs like the eye, rib, lung, spinal cord, spleen, liver, and kidney, as well as vertebrae. For each shape class, 16 CT images were selected from an archive. These images were divided into two trial sets for training and testing models. Hand-drawn contours from a previous study were used as ground truth.

Key Insights and Ideas

  • Shape knowledge significantly reduces manual effort: Both the min/max and probabilistic models, even when used without edge data to update them, drastically reduced the number of user-defined radials needed for segmentation compared to manual tracing.
  • Probabilistic model's flexibility is a double-edged sword: While the probabilistic model's ability to continuously update radial distributions offers potential for overcoming edge detection errors, its computational complexity is significantly higher than the min/max model.
  • Edge detection strategy is critical: The paper suggests that errors in the edge detection process, particularly premature commitment to incorrect edges, can lead shape models to diverge from the correct boundaries. This highlights the importance of a robust edge detection and analysis strategy.
  • Sequential updates might not always be beneficial: Surprisingly, segmentations that did not use edge data to sequentially update the shape models performed as well as or better than those that did, and were significantly faster. This suggests that for some shapes, a good initial shape prior is sufficient.

Key Results

  • Reduced User Interaction: The min/max and probabilistic models required significantly fewer user-defined radials (8.7 and 7.9 on average, respectively) compared to a control model without shape knowledge (11.7) and manual segmentation (24.0).
  • Speed Improvements (with caveats): While the min/max model was comparable in speed to manual segmentation, the probabilistic model was considerably slower (166.2 seconds vs. 20.8 seconds for min/max). However, when sequential updates were disabled, both models showed substantial speedups (8.3 seconds for min/max, 5.4 seconds for probabilistic) compared to manual segmentation (27.9 seconds).
  • Shape-Specific Performance: The models performed exceptionally well on structures like the eye, spinal cord, ribs, and vertebrae, with few errors. Performance was less consistent on the liver and lungs, attributed to high variability in liver models and insufficient variation in lung models, respectively.

Limitations and Future Work

  • Computational Cost of Probabilistic Model: The current implementation of the probabilistic model is computationally expensive, primarily due to its Lisp-level implementation. Rewriting intensive routines in C and using sparse matrix methods are suggested improvements.
  • Edge Detection Strategy: The paper identifies the edge detection strategy as a potential bottleneck. Future work could involve more sophisticated edge analysis, such as considering the entire radial for edge detection and using weighted Gaussian distributions to model potential edges.
  • Model Variation: For some shapes, insufficient variation in the test models led to poor performance. Testing on the models' own training data helped isolate this issue.
  • Integration with Other Models: The authors suggest combining shape models with active contour models (snakes) to leverage the strengths of both, particularly for overcoming the initialization challenges of snakes.
  • 3D Extension: The current models are 2D, and extending them to handle 3D structures is a natural next step.

Summary updated

Abstract

Accurate image segmentation is one of the key problems in computer vision. In domains such as radiation treatment planning, dosimetrists must manually trace the outlines of a few critical structures on large numbers of images. Considerable similarity can be seen in the shape of these regions, both between adjacent slices in a particular patient and across the spectrum of patients. Consequently we should be able to model this similarity and use it to assist in the process of segmentation. Previous work has demonstrated that a constraint-based 2D radial model can capture generic shape information for certain shape classes, and can reduce user interaction by a factor of three over purely manual segmentation. Additional simulation studies have shown that a probabilistic version of the model has the potential to further reduce user interaction. This paper describes an implementation of both models in a general-purpose imaging and graphics framework and compares the usefulness of the models on several shape classes.
BibTeX
@inproceedings{hinshaw1995lt,
  author = {Kevin P. Hinshaw and Russ B. Altman and James F. Brinkley},
  title = {{\<title\>Shape-based models for interactive segmentation of medical images\</title\>}},
  booktitle = {Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE},
  year = {1995},
  doi = {10.1117/12.208750},
}
Figure