Abstract
Object skeleton extraction is a critical computer vision task that identifies the medial axes of objects. Unlike edge detection, which relies on local contrast, skeletonization requires high-level semantic understanding. This paper introduces the Multi-Scale Bidirectional Fully Convolutional Network (MSB-FCN), an architecture designed to leverage deep semantic features while discarding noisy low-level information. By employing a bidirectional, densely connected structure across multiple scales, MSB-FCN achieves state-of-the-art performance, notably improving accuracy by 28.7% on the challenging Sym-PASCAL dataset.
Introduction
Extracting the "inner bones" or skeletons of objects is fundamental for shape analysis and object recognition. While superficially similar to edge detection, skeleton extraction is distinct: edges are defined by local changes in color or texture, whereas skeletons represent the global geometric structure of an object.
Previous approaches often adapted edge detection architectures like Holistically-nested Edge Detection (HED). However, these models typically utilize "skip-layer" structures that mix shallow, low-level features with deep, semantic ones. The authors of this work argue that for skeletonization, shallow features act as "messy" noise, particularly in complex scenes. MSB-FCN addresses this by focusing exclusively on high-level features processed through a novel multi-scale bidirectional framework.
Methodology: The MSB-FCN Architecture
The MSB-FCN consists of three primary modules designed to capture multi-scale semantic information without the interference of low-level noise.
- Feature Pyramid Construction: Instead of aggregating features from all network layers, MSB-FCN extracts the final, most semantic layer from a pre-trained ResNet-101. Pyramid pooling then generates four distinct scales (10x10, 20x20, 30x30, and 60x60 bins). This allows the model to analyze object structure at various resolutions using only "smart" high-level features.
- Multi-Scale Bidirectional Network: Features are processed in two directions: coarse-to-fine (global context to local detail) and fine-to-coarse (local detail to global context). This bidirectional flow ensures that predictions at any scale are informed by both the overall object shape and its specific parts.
- Dense Connections and Fusion: Inspired by DenseNets, every scale is linked to every other scale in a feed-forward manner. This dense connectivity prevents information loss during multi-scale processing. Finally, a Fusion Module learns the optimal weights to combine these multi-scale predictions into a single, high-resolution skeleton map.
Experimental Evaluation
The model was rigorously tested on four major benchmarks:
- SK-SMALL: Known for high variance in object structure.
- WH-SYMMAX: Focused on the Weizmann Horse dataset.
- SK-LARGE: A comprehensive, large-scale benchmark.
- Sym-PASCAL: The most demanding dataset, featuring 20 object classes in complex, "in-the-wild" environments.
Results and Discussion
MSB-FCN consistently outperformed both traditional and deep-learning-based predecessors. On the Sym-PASCAL dataset, it achieved an F-measure of 0.570, representing a 28.7% improvement over the previous state-of-the-art (SRN at 0.443).
Key insights from the study include:
- Efficacy of Deep Features: Shallow CNN layers are not only unnecessary but can be detrimental to skeleton detection. By relying on the final convolutional block, the model avoids confusion from background textures.
- Bidirectional Context: Skeletons are intrinsically linked to object boundaries. The bidirectional flow allows the network to better model the relationship between global shape and local symmetry.
- Efficiency: The model processes images in approximately 0.13 seconds on a modern GPU, making it viable for near-real-time applications.
Conclusion
The MSB-FCN demonstrates that a focused, multi-scale approach using high-level semantic features is superior for object skeleton extraction. While the model excels in cluttered scenes, it still faces challenges in extreme cases where objects are nearly indistinguishable from their backgrounds. Future work will explore applying this bidirectional architecture to other high-level vision tasks such as 3D structure estimation.
Summary updated