Abstract
Curvilinear Structure Segmentation (CSS) is critical for applications ranging from medical diagnostics to industrial inspection. However, existing models often struggle with domain gaps and the inherent difficulty of capturing thin, low-contrast structures. We present UCS, a universal framework that adapts the Segment Anything Model (SAM) for CSS. By employing a Sparse Adapter strategy and automated frequency-domain prompting, UCS achieves state-of-the-art performance across medical, engineering, and natural imagery using a single set of model weights, requiring only 2.65% trainable parameters.
Introduction
Curvilinear Structure Segmentation involves identifying thin, intertwined objects such as retinal vessels, pavement cracks, or leaf veins. While these structures share a common "line-like" geometry, computer vision models typically face two major hurdles:
- The Domain Gap: Models trained on specific datasets (e.g., medical imaging) often fail when applied to different domains (e.g., infrastructure cracks) due to variations in background and texture.
- The "Thinness" Problem: Foundation models like SAM are optimized for general objects with distinct boundaries and often fragment or overlook slender, low-contrast curvilinear structures.
UCS addresses these challenges by providing a universal architecture capable of generalizing across diverse domains without task-specific retraining.
Methodology
UCS is built upon the pretrained SAM image encoder, enhanced by three key architectural innovations designed specifically for curvilinear geometry:
- Sparse Adapter (SA): Rather than inserting adapters into every transformer block, UCS utilizes a "less is more" approach. Adapters are placed only at specific intervals (the 5th, 11th, 17th, and 23rd blocks). This sparsity prevents overfitting to specific textures and preserves SAM’s broad pretrained knowledge.
- Prompt Generation (PG) Module: To automate the prompting process, UCS uses a Fast Fourier Transform (FFT) with a high-pass filter. This module extracts high-frequency curve-specific cues directly from the image, providing the encoder with automated structural prompts.
- Dual-Compression Mask Decoder: This component ensures fine-detail retention through Hierarchical Feature Compression (HFC), which aggregates multi-layer features, and Guidance Feature Compression (GFC), which uses Principal Component Analysis (PCA) to identify dominant structural directions.
Experimental Setup and Datasets
The model was trained on a diverse corpus of 3,244 images spanning medical (retinal, nerve, X-ray) and engineering (pavement cracks) domains. Evaluation was conducted on seven public benchmarks and a new in-house test dataset covering eight natural categories: branch, crack, floor, scratch, soil, wire, leaf, and tyre.
Key Results and Discussion
UCS demonstrates significant performance gains over both domain-specific CNNs and general-purpose foundation models.
- Superior Generalization: UCS achieved an F1-score of 69.31 on the "Crack" category of the in-house test set, significantly outperforming HQ-SAM (54.45).
- Zero-Shot Capability: The model showed exceptional performance on unseen categories, such as leaf veins and tyre treads, highlighting its robust structural logic.
- Efficiency: By making only 2.65% of parameters trainable, UCS remains highly efficient for fine-tuning compared to full-model adaptation.
Key Insights
- Sparsity as a Regularizer: The finding that fewer adapters lead to better generalization suggests that dense adaptation can lead to catastrophic forgetting of the foundation model's universal features.
- Frequency-Domain Shortcuts: Utilizing high-pass FFT filters provides a mathematically grounded method to isolate thin structures, reducing the model's reliance on learning every possible curve variation from scratch.
Conclusion and Future Work
UCS represents a major step toward a universal solution for curvilinear segmentation. While highly effective, future research will focus on distilling the large SAM backbone into a more lightweight version and exploring multi-modal fusion (e.g., incorporating depth or infrared data) to further enhance robustness in complex industrial environments.
Summary updated