Abstract
This paper introduces a robust deep-learning framework for object skeleton extraction from natural images. By leveraging a multi-task learning approach that combines pixel classification with scale regression and employing a novel variable coefficient loss function, the proposed method addresses extreme class imbalance and scale ambiguity. Experimental results on benchmarks like SK-LARGE demonstrate state-of-the-art performance in localization accuracy and topological preservation.
Introduction
Skeleton extraction involves identifying the "symmetry axis" of an object—a thin, simplified version of its shape that preserves its fundamental topology. This process is crucial for various computer vision applications, including gesture recognition, medical imaging, and object detection. However, performing this task in "natural images" (real-world photos with complex backgrounds) is notoriously difficult due to two primary factors:
- Extreme Class Imbalance: Skeleton pixels typically constitute less than 1% of the total pixels in an image, causing standard networks to favor background classification.
- Scale Ambiguity: Accurately locating the skeleton requires an understanding of the object's local mass or "thickness."
Methods
The researchers developed a solution based on a Fully Convolutional Network (FCN) architecture using a VGG16 backbone. The approach introduces three major innovations:
Hierarchical Integration
The network captures features at five different stages, ranging from fine details in shallow layers to global context in deeper layers. A hierarchical integration mechanism fuses these features across four levels (H1–H4), allowing the model to refine its predictions iteratively.
Multi-Task Learning (Classification + Regression)
Instead of treating skeleton extraction as a simple binary classification problem, the network simultaneously predicts the radius of the object at each point. By adding a regression branch to predict scale, the model gains a better geometric understanding, which significantly improves the localization of the skeleton pixels.
Variable Coefficient Loss Function
To address the class imbalance problem, the authors modified the standard Softmax loss by introducing a dynamic weight ($a_i$) for each class based on its frequency. This ensures that rare skeleton features of varying thicknesses are not overwhelmed by the background during training.
Experimental Results
The model was evaluated on three public datasets: SK-LARGE, SK-SMALL, and WH-SYMMAX. The proposed method outperformed previous state-of-the-art (SOTA) models such as FSDS and Hi-Fi. On the SK-LARGE dataset, the model achieved a record F-measure of 0.6922.
Qualitative analysis shows that the model successfully extracts clean, connected skeletons even in images with high clutter or low contrast. Furthermore, cross-dataset testing demonstrated the model's superior generalization capabilities, suggesting it learns fundamental geometric principles rather than dataset-specific patterns.
Conclusion and Future Work
The combination of multi-task learning and a variable coefficient loss function provides a powerful framework for skeleton extraction in complex scenes. While the model shows significant improvements, limitations remain regarding sample scarcity for rare object types and background ambiguity in highly textured scenes. Future research will focus on expanding training diversity and further refining the regression branch to achieve human-level perception accuracy.
Summary updated