TL;DR
This paper introduces DexiNed, a novel deep learning model for edge detection that generates thin, human-perceptible edge maps without requiring pre-trained weights. The authors also present BIPED, a new, carefully annotated dataset for edge detection, and demonstrate DexiNed's superior generalization capabilities across various datasets compared to existing state-of-the-art methods.
What Problem Does This Paper Address?
Edge detection is a fundamental task in computer vision, crucial for applications ranging from image segmentation and recognition to more advanced tasks like image-to-image translation and photo sketching. While traditional methods like Canny have been around for decades, and more recent deep learning approaches have shown promise, several challenges remain. Many existing deep learning edge detectors are trained on datasets primarily designed for object boundary or contour detection. This can lead to missed edges in the final output, as these datasets may not comprehensively capture all visual edges present in an image. Furthermore, the generalizability of these models to unseen datasets, especially those specifically curated for edge detection, is often limited. The authors aim to address these issues by proposing a robust CNN architecture that generates accurate, thin edge maps and can generalize well across different datasets without extensive fine-tuning.
Methods
The core of the proposed method is the DexiNed architecture, which is inspired by the Xception network but with significant modifications. DexiNed employs a dense inception-like structure within its encoder, where each block is densely interconnected with the outputs of previous blocks. This design aims to preserve edge features throughout the network, mitigating the loss that can occur in deeper layers.
Key architectural features include:
- Dense Connections: Unlike Xception's separable convolutions, DexiNed uses standard convolutions. Crucially, outputs from earlier blocks are fed into later blocks, creating a dense flow of information.
- Multi-Scale Feature Extraction: The network generates intermediate edge maps at various scales through upsampling blocks. These are then fused to produce the final edge map.
- Skip Connections: DexiNed incorporates two types of skip connections, inspired by ResNet and RDN, to further enhance feature propagation and prevent gradient vanishing.
- Upsampling Blocks (UB): These blocks are responsible for generating intermediate edge maps from the features extracted by the Dexi blocks. They utilize a stack of convolutional and transposed convolutional filters. The authors explored different upsampling strategies (bi-linear interpolation, subpixel convolution, transpose convolution) and found that a specific transpose convolution approach (DexiNed-dc) yielded the best results.
- Training from Scratch: A notable aspect is that DexiNed is trained from scratch without using any pre-trained weights, emphasizing its ability to learn edge features effectively from the ground up.
Datasets
The paper introduces a new dataset and utilizes several existing ones for training and evaluation:
- BIPED (Barcelona Images for Perceptual Edge Detection): This is the primary dataset introduced by the authors. It comprises 250 high-resolution outdoor images (1280x720 pixels) carefully annotated by computer vision experts. It's intended as a benchmark specifically for edge detection. The authors also detail a data augmentation process that significantly increases the training set size.
- MDBD (Multicue Dataset for Boundary Detection): Used for evaluation, this dataset is intended for psychophysical studies on object boundary detection.
- CID (Contour Detection): Used for evaluation, this dataset contains images with ground truth edges and object contours.
- BSDS (Berkeley Segmentation Dataset): Both BSDS300 and BSDS500 are used for evaluation. These datasets are primarily for image segmentation and boundary detection.
- NYUD (New York University Dataset): A dataset of RGBD images intended for segmentation.
- PASCAL (Pascal-Context): A popular dataset for segmentation, often used for edge and boundary detection.
It's important to note that while DexiNed is trained on BIPED, it is evaluated on other datasets without further fine-tuning, highlighting its generalization capabilities.
Key Insights and Ideas
- Thin Edge Maps are Crucial: The paper emphasizes the importance of generating "thin" edge maps that are visually pleasing and accurate, a goal DexiNed aims to achieve.
- Edge Detection vs. Boundary Detection: A key insight is the distinction between general edge detection and object boundary/contour detection. Datasets optimized for the latter may not be ideal for training a general edge detector, leading to missed edges.
- Generalization is Key: DexiNed's ability to perform well on various datasets, even those it wasn't trained on, demonstrates the robustness of its learned edge features. This is a significant advantage over models that might overfit to specific dataset characteristics.
- Training from Scratch is Viable: The success of DexiNed trained from scratch challenges the common practice of relying heavily on pre-trained models, suggesting that well-designed architectures can learn effectively from raw data.
- Multi-Scale Fusion for Robustness: The architecture's multi-scale approach, combining features from different network depths and then fusing them, contributes to capturing edges at various scales and complexities.
Key Results
- Quantitative Performance: DexiNed-a (averaged predictions) achieved the best quantitative results across multiple evaluation metrics (ODS, OIS, AP) on the BIPED dataset, outperforming other state-of-the-art methods. DexiNed-f (fused predictions) also performed very competitively.
- Generalization: On the MDBD dataset, DexiNed achieved the best performance. While it didn't always achieve the top scores on datasets primarily for segmentation (like BSDS, NYUD, PASCAL), the paper argues this is due to the nature of those datasets, which penalize general edge detection. When images are fully annotated, DexiNed's performance is strong.
- Qualitative Results: Figures 5 and 7 showcase DexiNed's ability to produce visually appealing, thin edge maps that are comparable to or better than other leading methods, even when those methods were trained on the same datasets.
Limitations and Future Work
- Limitations: The paper acknowledges that when evaluating DexiNed on datasets not specifically designed for edge detection (e.g., segmentation datasets), its performance might not reach the absolute top scores. This is attributed to the inherent differences in what these datasets aim to capture.
- Future Work: The authors suggest that their proposed architecture and approach could be extended to tackle contour and boundary detection problems more directly. Further research could also explore optimizing the upsampling process and exploring different fusion strategies.
Summary updated