TL;DR
While deep learning has pushed edge detection to human-level performance, most models produce "blurry" results that hinder downstream tasks. This paper introduces Crisp Edge Detection (CED), a novel architecture using a top-down refinement pathway and sub-pixel convolution to produce precisely localized, sharp boundaries that significantly boost performance in optical flow, object proposals, and semantic segmentation.
What Problem Does This Paper Address?
In the world of computer vision, edge detection is a fundamental building block. For years, the field celebrated models like the Holistic Edge Detector (HED), which achieved "human-level" scores on standard benchmarks. However, the authors of this paper point out a hidden flaw: these models are "vaguely right" but "precisely wrong."
Standard evaluation metrics (like the ODS score on BSDS500) allow for a small amount of slack—an edge pixel is considered "correct" if it's within a certain distance ($d$) of the ground truth. Deep Convolutional Networks (ConvNets) exploit this by producing thick, blurry probability maps. While these maps capture the existence of an edge, they fail to capture its precise location. This blurriness is a major problem for "higher-level" tasks like optical flow or object proposal generation, which require pixel-perfect boundaries to function correctly.
Methods
The authors propose a new architecture called Crisp Edge Detection (CED). It builds upon the standard feed-forward ConvNet (like VGG or ResNet) but adds several critical components to "sharpen" the output:
- Backward-Refining Pathway: Instead of simply fusing multi-scale features at the end, CED uses a top-down pathway. It takes the high-level semantic information from the deepest layers and progressively fuses it with the fine-grained spatial details of the shallower layers.
- Sub-pixel Convolution: This is the "secret sauce." Traditional models use bilinear interpolation or deconvolution to upsample feature maps, which inherently introduces blur. CED uses sub-pixel convolution (originally designed for super-resolution), which rearranges multiple feature channels into a higher-resolution map, preserving sharpness and adding non-linearity.
- Large Kernels for Side Outputs: To better capture diverse transformations and ensure dense connections between feature maps, the model adds extra convolutional layers with increasing kernel sizes (up to 7x7) at each side output.
Datasets
The model was trained and evaluated on several key benchmarks:
- BSDS500: The standard benchmark for boundary detection.
- PASCAL-Context: A larger-scale dataset used to test the model's ability to generalize to 60 different object and "stuff" categories.
- MPI Sintel: Used to evaluate the impact of crisp edges on optical flow estimation.
- Pascal VOC 2012: Used to evaluate object proposal generation.
Key Insights and Ideas
- The "Crispness" Test: The authors proved that standard models are poorly localized by tightening the evaluation criteria. When the permissible matching distance was reduced from $d$ to $d/4$, the performance of standard models like HED plummeted, while CED remained robust.
- Architecture Matters for Localization: The blurriness in deep models isn't just a side effect; it's "deeply rooted" in the architecture—specifically the successive pooling layers that discard spatial resolution.
- Downstream Synergy: Edge detection shouldn't be evaluated in a vacuum. The paper demonstrates a clear "trickle-down" effect: better (sharper) edges lead to better performance in complex tasks like semantic segmentation without needing to re-train the segmentation model itself.
Key Results
- State-of-the-Art Accuracy: On the BSDS500 dataset, the ResNet-based version of CED achieved an ODS score of 0.810 (and 0.822 with extra VOC data), surpassing human performance (0.803) and previous state-of-the-art methods.
- Optical Flow: Using CED edges in the EpicFlow algorithm reduced the Average Endpoint Error (AEE) on the Sintel dataset from 3.588 to 3.549.
- Object Proposals: CED significantly improved the quality of object proposals. Using CED with the MCG algorithm achieved a 0.73 Average Recall with roughly 2,650 proposals, a significant efficiency gain over baseline edge detectors.
- Semantic Segmentation: When used as a post-processing step (via Boundary Neural Fields), CED edges improved the Mean IOU of DeepLab on PASCAL-Context from 42.6 to 44.5.
Limitations and Future Work
The authors acknowledge that while CED produces much sharper edges than previous deep models, perfectly localizing every pixel in complex natural scenes remains an open challenge. The current method relies on a pre-trained backbone (like ResNet); future work could involve designing architectures specifically optimized for edge detection from the ground up. Additionally, while the paper shows benefits for several tasks, the authors suggest that integrating "crispness" directly into the loss functions of end-to-end high-level vision models could yield even greater gains.
Summary updated