Deep Crisp Boundaries: From Boundaries to Higher-level Tasks

Yupei Wang, Xin Zhao, Yin Li, Kaiqi Huang

Abstract

Edge detection has made significant progress with deep Convolutional Networks (ConvNets), approaching human-level performance on standard benchmarks. However, a systematic study reveals that these detectors often fail to accurately localize edge pixels, producing "blurry" results that are suboptimal for downstream tasks. This paper introduces a novel refinement architecture for learning a Crisp Edge Detector (CED). By leveraging a top-down backward refinement pathway and sub-pixel convolution, CED progressively increases feature map resolution to generate precisely localized boundaries. Our method surpasses human accuracy on BSDS500 and demonstrates significant performance gains in optical flow estimation, object proposal generation, and semantic segmentation.

The Localization Problem in Edge Detection

While models like the Holistic Edge Detector (HED) achieve high scores on standard benchmarks, they often produce thick, probability-based maps rather than sharp boundaries. Standard evaluation metrics, such as the ODS score on BSDS500, allow for a matching distance ($d$) between predicted and ground truth pixels. ConvNets exploit this slack, becoming "vaguely right" about an edge's existence but "precisely wrong" about its exact location.

Figure 2: (a) Visual comparison of edge detectors. HED produces thick, blurry boundaries compared to Ground Truth. (b) F1-Score performance drops significantly for standard models as the matching distance threshold is tightened from d to d/4.

As shown in Figure 2, when the permissible matching distance is reduced to $d/4$, the performance of traditional deep detectors like HED plummets, highlighting their poor localization capabilities.

Proposed Method: Crisp Edge Detection (CED)

To address the blurriness inherent in standard ConvNet architectures, the authors propose the CED framework, which introduces three key innovations:

  1. Backward-Refining Pathway: Unlike simple multi-scale fusion, CED employs a top-down pathway that integrates high-level semantic information from deep layers with fine-grained spatial details from shallower layers.
  2. Sub-pixel Convolution: To avoid the blur introduced by bilinear interpolation or deconvolution, CED uses sub-pixel convolution. This technique rearranges feature channels into a higher-resolution map, preserving sharpness while adding non-linearity.
  3. Enhanced Side Outputs: The model incorporates additional convolutional layers with large kernels (up to 7x7) at each side output to capture diverse transformations and ensure dense feature connectivity.

Key Insights: Why Sharpness Matters

The blurriness in deep models is a direct consequence of successive pooling layers that discard spatial resolution for the sake of semantic abstraction. This lack of precision is particularly detrimental for higher-level vision tasks.

Figure 6: A zoomed-in view of a deer shows that HED produces thick, poorly localized edges before Non-Maximum Suppression (NMS), which can mislead downstream algorithms.

As illustrated in Figure 6, the "thick" edges produced by standard models fail to capture the delicate contours of objects, such as the deer's antlers. CED's ability to maintain "crispness" ensures that boundaries are accurately placed, providing a more reliable input for subsequent processing.

Experimental Results and Downstream Impact

CED was evaluated across several benchmarks, demonstrating state-of-the-art performance and clear benefits for downstream applications:

  • Boundary Detection: On BSDS500, the ResNet-based CED achieved an ODS score of 0.810 (0.822 with VOC data), surpassing the human benchmark of 0.803.
  • Optical Flow: Integrating CED edges into EpicFlow reduced the Average Endpoint Error (AEE) on the MPI Sintel dataset from 3.588 to 3.549.
  • Object Proposals: Using CED with the MCG algorithm significantly improved proposal quality, reaching a 0.73 Average Recall with fewer proposals than baseline methods.
  • Semantic Segmentation: As a post-processing step via Boundary Neural Fields, CED edges improved the Mean IOU of DeepLab on PASCAL-Context from 42.6 to 44.5.

Conclusion

Deep Crisp Boundaries demonstrates that the quality of edge detection is not just about detection probability, but also about precise localization. By introducing a refinement architecture that prioritizes sharpness, the authors have bridged the gap between low-level edge detection and high-level vision tasks, proving that better boundaries lead to better overall scene understanding.

Summary updated

Abstract

Edge detection has made significant progress with the help of deep Convolutional Networks (ConvNet). These ConvNet based edge detectors have approached human level performance on standard benchmarks. We provide a systematical study of these detectors' outputs. We show that the detection results did not accurately localize edge pixels, which can be adversarial for tasks that require crisp edge inputs. As a remedy, we propose a novel refinement architecture to address the challenging problem of learning a crisp edge detector using ConvNet. Our method leverages a top-down backward refinement pathway, and progressively increases the resolution of feature maps to generate crisp edges. Our results achieve superior performance, surpassing human accuracy when using standard criteria on BSDS500, and largely outperforming state-of-the-art methods when using more strict criteria. More importantly, we demonstrate the benefit of crisp edge maps for several important applications in computer vision, including optical flow estimation, object proposal generation and semantic segmentation.
BibTeX
@article{wang2019deep,
  author = {Yupei Wang and Xin Zhao and Yin Li and Kaiqi Huang},
  title = {{Deep Crisp Boundaries: From Boundaries to Higher-level Tasks}},
  journal = {IEEE Trans. Image Process.},
  year = {2019},
  url = {https://openreview.net/forum?id=SVui_sK5a8jM},
}
Figure