SDC - Stacked Dilated Convolution: A Unified Descriptor Network for Dense Matching Tasks

René Schuster, Oliver Wasenmüller, Christian Unger, Didier Stricker

Abstract

Dense pixel matching is important for many computer vision tasks such as disparity and flow estimation. We present a robust, unified descriptor network that considers a large context region with high spatial variance. Our network has a very large receptive field and avoids striding layers to maintain spatial resolution. These properties are achieved by creating a novel neural network layer that consists of multiple, parallel, stacked dilated convolutions (SDC). Several of these layers are combined to form our SDC descriptor network. In our experiments, we show that our SDC features outperform state-of-the-art feature descriptors in terms of accuracy and robustness. In addition, we demonstrate the superior performance of SDC in state-of-the-art stereo matching, optical flow and scene flow algorithms on several famous public benchmarks.

Introduction

High-level computer vision tasks, including autonomous driving and robotic navigation, depend heavily on dense matching—the process of identifying corresponding pixels across multiple images. This task is inherently challenging due to factors like varying illumination, sensor noise, and occlusions.

Effective dense matching requires descriptors that balance two competing needs:

  1. Large Context: Robustness requires a large receptive field to capture neighborhood information.
  2. High Precision: Accuracy requires maintaining fine-grained spatial resolution.

Traditional deep learning models often sacrifice resolution for context by using striding or pooling layers. SDC (Stacked Dilated Convolution) addresses this by providing a massive receptive field (81x81 pixels) without any downsampling, ensuring zero loss in spatial resolution.

Methods: The SDC Architecture

The fundamental building block of this approach is the SDC layer. Unlike standard convolutional layers, an SDC layer employs four parallel dilated convolutions with varying dilation rates (1, 2, 3, and 4) and 5x5 kernels.

The SDC layer architecture uses parallel dilated convolutions with varying rates to capture multi-scale context without losing resolution.

Key Architectural Features

  • Parallel Dilated Convolutions: By using "dilated" filters (filters with gaps), the network expands its receptive field without increasing parameter count or computational overhead.
  • Concatenation over Summation: Instead of summing the outputs of parallel branches, SDC concatenates them. This preserves multi-scale information throughout the network.
  • Stacking for Resolution: The full network consists of five stacked SDC layers. By avoiding striding and pooling, the output feature map maintains the exact dimensions of the input image.
  • Mitigating the "Gridding Effect": Standard dilated sequences can suffer from "gridding," where only a sparse subset of pixels is sampled. SDC’s parallel structure with different rates ensures every pixel in the receptive field contributes to the final descriptor.

The network is trained using a triplet loss function, which encourages the model to minimize the distance between matching patches (reference and positive) while maximizing the distance to non-matching ones (negative).

Experiments and Results

To ensure the descriptor's universality, it was trained on a diverse union of datasets including KITTI 2015 (automotive), MPI Sintel (complex CGI), Middlebury (indoor stereo), HD1K (optical flow), and ETH3D (two-view stereo).

Performance Highlights

SDC demonstrates significant improvements across various benchmarks:

  • Accuracy: SDC achieved a 97.2% accuracy in patch triplet matching, surpassing previous leaders like L2Net (96.7%) and traditional methods like SIFT (89.0%).
  • Algorithmic Integration: When integrated into the FlowFields++ algorithm, SDC reduced matching outliers by up to 50%.
  • Match Density: In scene flow (SFF) tasks on the KITTI benchmark, SDC increased the density of valid matches from 43.6% to 67.0%.
Visual comparison showing how SDC features improve the density and accuracy of existing matching algorithms like ELAS, CPM, and SFF.

Conclusion and Future Work

While SDC offers a powerful "drop-in" replacement for traditional descriptors, it does have limitations. It shows slightly lower performance on dynamic foreground objects in the KITTI dataset, likely due to their underrepresentation in training data. Additionally, while efficient for a deep network, it remains more computationally intensive than simple bit-wise descriptors like CENSUS.

Future research will focus on re-tuning classic matching algorithms to better leverage the high-dimensional discriminative power of SDC features and improving performance in dynamic foreground regions.

Summary updated

Abstract

Dense pixel matching is important for many computer vision tasks such as disparity and flow estimation. We present a robust, unified descriptor network that considers a large context region with high spatial variance. Our network has a very large receptive field and avoids striding layers to maintain spatial resolution. These properties are achieved by creating a novel neural network layer that consists of multiple, parallel, stacked dilated convolutions (SDC). Several of these layers are combined to form our SDC descriptor network. In our experiments, we show that our SDC features outperform state-of-the-art feature descriptors in terms of accuracy and robustness. In addition, we demonstrate the superior performance of SDC in state-of-the-art stereo matching, optical flow and scene flow algorithms on several famous public benchmarks.
BibTeX
@inproceedings{schuster2019sdc,
  author = {René Schuster and Oliver Wasenmüller and Christian Unger and Didier Stricker},
  title = {{SDC - Stacked Dilated Convolution: A Unified Descriptor Network for Dense Matching Tasks}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2019},
  url = {https://openreview.net/forum?id=Q-Vs5yG6es7},
}
Figure