TL;DR
This paper introduces "Scanner," an interactive system that uses shape models to assist in medical image segmentation, significantly reducing manual effort. They compare two shape models, a "min/max" and a "probabilistic" model, finding that both improve segmentation efficiency, with the probabilistic model showing promise but facing computational challenges.
What Problem Does This Paper Address?
Accurate segmentation of medical images is crucial for applications like radiation treatment planning, where clinicians must manually trace critical structures on numerous images. This process is time-consuming and prone to error, especially when adjacent soft tissues have similar densities, making traditional edge detection unreliable. The paper addresses the need for a more automated and efficient segmentation approach by leveraging the inherent shape similarities of anatomical structures across different images and patients.
Methods
The core of the paper is the "Scanner" system, which implements two types of shape models:
- Min/max model: This model represents shape using uncertainty intervals for radial lengths from a contour's center. It also incorporates a matrix of "shape constraints" that define acceptable ratios between radial lengths, ensuring that the overall contour maintains a consistent shape. Constraint propagation is used to tighten these intervals as more information becomes available.
- Probabilistic model: This model uses probability distributions (mean and variance) for each radial's length. It also captures covariation between radial lengths. This approach allows for more flexible updates to radial lengths, as it doesn't fix a value once assigned. The Kalman filter is used to update these distributions as edge data is incorporated.
Both models are integrated into the "Scanner" system, which provides an interactive interface for users to guide the segmentation process. Users define a contour's initial orientation and scale, and the system then uses the shape model to guide an edge detector. Users can also manually correct segmentation errors.
Datasets
The paper describes an evaluation using fifteen biological shape classes, representing cross-sections of organs like the eye, rib, lung, spinal cord, spleen, liver, and kidney, as well as vertebrae. For each shape class, 16 CT images were selected from an archive. These images were divided into two trial sets for training and testing models. Hand-drawn contours from a previous study were used as ground truth.
Key Insights and Ideas
- Shape knowledge significantly reduces manual effort: Both the min/max and probabilistic models, even when used without edge data to update them, drastically reduced the number of user-defined radials needed for segmentation compared to manual tracing.
- Probabilistic model's flexibility is a double-edged sword: While the probabilistic model's ability to continuously update radial distributions offers potential for overcoming edge detection errors, its computational complexity is significantly higher than the min/max model.
- Edge detection strategy is critical: The paper suggests that errors in the edge detection process, particularly premature commitment to incorrect edges, can lead shape models to diverge from the correct boundaries. This highlights the importance of a robust edge detection and analysis strategy.
- Sequential updates might not always be beneficial: Surprisingly, segmentations that did not use edge data to sequentially update the shape models performed as well as or better than those that did, and were significantly faster. This suggests that for some shapes, a good initial shape prior is sufficient.
Key Results
- Reduced User Interaction: The min/max and probabilistic models required significantly fewer user-defined radials (8.7 and 7.9 on average, respectively) compared to a control model without shape knowledge (11.7) and manual segmentation (24.0).
- Speed Improvements (with caveats): While the min/max model was comparable in speed to manual segmentation, the probabilistic model was considerably slower (166.2 seconds vs. 20.8 seconds for min/max). However, when sequential updates were disabled, both models showed substantial speedups (8.3 seconds for min/max, 5.4 seconds for probabilistic) compared to manual segmentation (27.9 seconds).
- Shape-Specific Performance: The models performed exceptionally well on structures like the eye, spinal cord, ribs, and vertebrae, with few errors. Performance was less consistent on the liver and lungs, attributed to high variability in liver models and insufficient variation in lung models, respectively.
Limitations and Future Work
- Computational Cost of Probabilistic Model: The current implementation of the probabilistic model is computationally expensive, primarily due to its Lisp-level implementation. Rewriting intensive routines in C and using sparse matrix methods are suggested improvements.
- Edge Detection Strategy: The paper identifies the edge detection strategy as a potential bottleneck. Future work could involve more sophisticated edge analysis, such as considering the entire radial for edge detection and using weighted Gaussian distributions to model potential edges.
- Model Variation: For some shapes, insufficient variation in the test models led to poor performance. Testing on the models' own training data helped isolate this issue.
- Integration with Other Models: The authors suggest combining shape models with active contour models (snakes) to leverage the strengths of both, particularly for overcoming the initialization challenges of snakes.
- 3D Extension: The current models are 2D, and extending them to handle 3D structures is a natural next step.
Summary updated