TL;DR
Fully Convolutional Networks (FCNs) excel at pixel-wise segmentation but often struggle to capture the global, anatomically plausible shapes of objects. This paper introduces a novel "star shape prior" regularization term directly into the FCN loss function to penalize non-star-shaped predictions. Applied to skin lesion segmentation, this end-to-end trainable approach achieved state-of-the-art results, securing first place in the ISBI 2017 challenge without relying on complex post-processing.
What Problem Does This Paper Address?
Semantic segmentation of medical images—such as isolating a skin lesion from healthy tissue for melanoma detection—is challenging due to massive variations in color, texture, and lighting.
Historically, traditional energy-based segmentation methods (like active contours or graph cuts) handled this by incorporating shape priors. If you know a skin lesion is generally a cohesive, roundish blob, you can mathematically constrain the algorithm to only output shapes fitting that description.
However, modern deep learning models like Fully Convolutional Networks (FCNs) typically rely on pixel-wise loss functions (e.g., binary cross-entropy). These loss functions evaluate each pixel independently, completely ignoring the global structure of the object. As a result, FCNs can produce fragmented, hole-filled, or anatomically impossible segmentation masks. The challenge this paper tackles is how to encode a global geometric rule—specifically a "star shape" prior—directly into the training of a deep neural network.
Methods
The authors propose augmenting the standard binary cross-entropy loss of an FCN with a novel regularization term: the star shape regularized loss ($L_{sh}$).
What is a star shape? An object is "star-shaped" if, for any point inside the object, the straight line connecting that point to the object's center lies entirely within the object. (Think of it as every part of the object having a direct "line of sight" to the center).
How it works: 1. Training: During training, the network automatically calculates the center of the object using the ground truth mask. The custom loss function samples lines radiating from this center to the edges of the image. If the network predicts a foreground pixel on one of these lines, but predicts a background pixel closer to the center on that same line, it violates the star shape rule. The loss function heavily penalizes these specific structural violations. 2. Inference: Because the prior is baked into the network's weights via the loss function during training, the network naturally learns to output star-shaped predictions. At inference time, the model requires no center-point input and no expensive post-processing (like Conditional Random Fields). It is a standard, fast forward-pass.
The authors tested this custom loss on two architectures: U-Net and ResNet-DUC.
Datasets
The method was trained and evaluated on the ISIC 2017 Skin Lesion Analysis Towards Melanoma Detection Challenge dataset.
- It contains 2,000 training images, 150 validation images, and 600 test images of dermoscopy scans.
- The authors verified the dataset's compatibility with their method, noting that only a negligible 0.14% of ground truth pixels in the entire dataset violated the star shape definition.
Key Insights and Ideas
- Priors belong in the loss function: You don't need complex probabilistic graphical models (like CRFs) or manual post-processing to enforce shape constraints. A cleverly designed loss function can force a network to learn global structural rules end-to-end.
- Zero inference-time overhead: Traditional graph-cut methods using shape priors often require a user to manually click the center of an object at runtime. By shifting the geometric constraint entirely to the training phase, the authors eliminated this bottleneck, allowing for fully automated, fast inference (1 second per image).
- Star-shape is a highly versatile prior: While it sounds restrictive, the star shape definition encompasses a massive variety of convex and semi-convex shapes, making it highly applicable to tumors, lesions, and many other biological structures.
Key Results
- Quantitative Boost: Adding the star shape prior improved the Jaccard index (Intersection over Union) by more than 3% across different architectures.
- State-of-the-Art Performance: The ResNet-DUC model trained with the star shape prior achieved a 77.3% Jaccard index, 85.7% Dice score, and 93.8% Accuracy.
- Challenge Winner: This approach secured the 1st rank among 21 participating teams in the ISBI 2017 skin segmentation challenge. Notably, the authors achieved this without the heavy ensembling or multi-scale inputs relied upon by the runner-up teams.
- Qualitative Improvements: Visually, the predictions generated with the star shape prior are much smoother and form a single, cohesive connected component, successfully eliminating the fragmented "noise" often seen in standard FCN outputs.
Limitations and Future Work
Limitations: While highly effective for skin lesions, the star shape prior inherently assumes the target object roughly conforms to a star-like topology. It would likely fail or require modification for highly irregular, branching, or ring-like structures (e.g., blood vessels or certain brain structures).
Future Work: The authors plan to extend this framework to incorporate other anatomically meaningful priors into FCNs. They also aim to apply these regularized networks to a broader range of 2D and 3D medical imaging applications where anatomical constraints are well understood.
Summary updated