Joint Topology-preserving and Feature-refinement Network for Curvilinear Structure Segmentation

Mingfei Cheng, Kaili Zhao, Xuhong Guo, Yajing Xu, Jun GuoICCV 2021 · DOI

Abstract

Curvilinear Structure Segmentation (CSS) is critical for analyzing thin, elongated features such as roads, cracks, and blood vessels. Standard deep learning models often struggle with topological discontinuities and blurred boundaries due to noisy backgrounds and down-sampling operations. We introduce the Joint Topology-preserving and Feature-refinement Network (JTFN), which employs an iterative feedback strategy and a dual-stream decoder to address these challenges. By integrating boundary detection as a geometric constraint and utilizing gated attention, JTFN achieves state-of-the-art performance in maintaining global connectivity and sharpening fine-grained details.

Introduction

Curvilinear structures are characterized by their high aspect ratios and often faint appearance against complex backgrounds. In domains like medical imaging and remote sensing, maintaining the topological integrity of these structures—ensuring a continuous road or vessel is not fragmented—is as crucial as pixel-wise accuracy. Traditional models frequently produce fragmented results or lose structural detail during down-sampling. JTFN proposes a unified approach that treats topology preservation and feature refinement as complementary objectives, leveraging an iterative process to progressively improve segmentation quality.

Methodology

JTFN enhances a standard U-Net architecture with a VGG-16 backbone through three primary innovations:

  • Iterative Feedback Strategy: The network processes inputs over multiple iterations (typically $T=3$), using previous predictions to guide subsequent refinement. This coarse-to-fine approach allows the model to progressively "heal" topological errors.
  • Feature Interactive Module (FIM): The decoder is bifurcated into two parallel streams: one for segmentation and one for boundary detection. The interaction between these streams enforces geometric constraints that preserve the structure's skeleton, as boundaries serve as a proxy for global topology.
  • Gated Attentive Unit (GAU): Positioned in skip connections, the GAU uses prior predictions as saliency maps. It supplements missing information in regions the model missed and enhances features in high-confidence regions to sharpen boundaries.

Experiments and Results

JTFN was evaluated on four diverse benchmarks: Crack500, CrackTree200, Massachusetts Roads, and DRIVE. These datasets represent the primary challenges in CSS, including complex backgrounds and varying structure widths.

Quantitative Performance

JTFN achieved state-of-the-art results across multiple metrics, including F1-score and topology-based measures. On the CrackTree200 dataset, it reached an F1-score of 84.19, significantly outperforming the baseline U-Net (78.86) and previous methods like DRU (80.49). In road extraction tasks, it achieved a 60.65 F1-score. Notably, the model showed superior topology preservation; on Crack500, the "Correctness" metric (measuring skeleton similarity) rose from 30.5 to 36.72.

Qualitative Analysis

Visual analysis confirms that JTFN effectively bridges gaps in fragmented structures and suppresses background noise. The combination of topology-preserving and feature-refining streams allows the model to produce continuous, sharp masks that closely match ground truth.

Qualitative comparison showing how the topology-preserving and feature-refinement streams contribute to the final JTFN prediction across various datasets.

Discussion and Conclusion

JTFN demonstrates that joint optimization of boundaries and masks can effectively preserve topology in curvilinear segmentation. While the iterative process increases computational cost—with inference time scaling linearly with the number of iterations—the gains in connectivity and detail are substantial. Future research will focus on adaptive loss weighting to further optimize the iterative refinement process and exploring robustness in datasets where boundaries are extremely ambiguous.

Summary updated

BibTeX
@inproceedings{cheng2021joint,
  author = {Mingfei Cheng and Kaili Zhao and Xuhong Guo and Yajing Xu and Jun Guo},
  title = {{Joint Topology-preserving and Feature-refinement Network for Curvilinear Structure Segmentation}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00706},
}
Figure