Abstract
Level set (LS)-based segmentation is a powerful tool in medical imaging, yet it often struggles with multi-instance objects in complex real-world scenes and exhibits high sensitivity to initialization parameters. This paper presents Contextual Recurrent Level Sets (CRLS), a framework that reformulates the classic variational level set method as a hidden state evolution within a Gated Recurrent Unit (GRU). By integrating fitting forces and contour length into the GRU's gating mechanism, the curve deformation process becomes learnable and differentiable. CRLS achieves state-of-the-art performance on PASCAL VOC 2012 and MS COCO while providing a massive speedup over traditional iterative solvers.
Introduction: The Challenge of Instance Segmentation
Instance segmentation—the task of identifying and outlining individual objects at the pixel level—remains a core challenge in computer vision. Historically, researchers utilized Level Set (LS) methods, which treat a contour as the zero-crossing of a higher-dimensional function that evolves toward object boundaries based on mathematical forces.
While LS methods are mathematically elegant, they face significant hurdles in "in-the-wild" scenarios:
- Sensitivity: They are highly sensitive to initial settings and the number of iterations.
- Complexity: They struggle to separate multiple instances of the same class (e.g., people in a crowd).
- Speed: Traditional iterative solvers are computationally expensive and slow.
The authors address these issues by promoting classic LS methods into a trainable deep learning framework.
Methodology: Contextual Recurrent Level Sets (CRLS)
The core innovation of this work is the Recurrent Level Set (RLS). The authors observed that the iterative nature of LS contour evolution mirrors the hidden state updates in a Recurrent Neural Network (RNN).
Recurrent Level Set (RLS)
By mapping the level set function ($\phi$) to the hidden state ($h_t$) of a GRU, the evolution process becomes differentiable and learnable via backpropagation. This allows the network to learn the optimal "path" for a contour to take toward an object boundary, rather than relying on fixed mathematical forces.
The CRLS Pipeline
To handle multiple instances effectively, the authors developed a three-stage pipeline:
- Detection Stage: Uses a VGG-16 backbone and a Region Proposal Network (RPN) to identify bounding boxes for potential objects.
- Segmentation Stage: For each box, the RLS (the GRU-based level set) evolves a mask. This "one-to-many" process takes the initial detection features and iteratively refines the object boundary.
- Classification Stage: A fully-connected network uses the generated mask and features to predict the object's class.
Key Insights
- Contour Evolution as Sequence Modeling: Treating time steps of a mathematical evolution as sequence steps in an RNN is a profound shift that enables end-to-end training.
- Efficiency through Learning: Traditional LS methods can take over 10 seconds to converge. By learning the weights, RLS performs the same task in milliseconds (0.008s).
- Contextual Integration: By combining RLS with a detection network, the model gains the "context" needed to apply level sets to specific instances, solving the multi-instance problem.
Results
The CRLS model demonstrated superior performance in both accuracy and speed:
- Accuracy: On PASCAL VOC 2012, CRLS achieved a 66.7% mAP (mean Average Precision), outperforming other state-of-the-art methods like MNC (63.5%) and SDS (49.7%).
- Speed: The RLS component is remarkably fast, reducing processing time from 13.5 seconds (baseline Chan-Vese) to just 0.008 seconds.
- Robustness: Qualitative results show the model successfully separating overlapping objects where traditional methods often merge them.
Conclusion and Future Work
CRLS successfully bridges the gap between classical variational methods and modern deep learning. While it shows strong results on standard benchmarks, challenges remain in extremely cluttered scenes with heavy occlusion, as seen in MS COCO results. Future work may extend this recurrent formulation to 3D medical imaging volumes, where traditional level sets are already a staple but suffer from similar speed and sensitivity issues.
Summary updated