Adaptive Unimodal Cost Volume Filtering for Deep Stereo Matching

Youmin Zhang, Yimin Chen, Xiao Bai, Jun Zhou, Kun Yu, Zhiwei Li, Kuiyuan Yang

Abstract

Deep stereo matching networks typically utilize a cost volume to represent matching costs across disparities. However, supervising only the final disparity regression leaves the cost volume under-constrained, often resulting in multi-modal or noisy distributions. We propose AcfNet (Adaptive Unimodal Cost Volume Filtering Network), which supervises the cost volume to follow a unimodal distribution. By integrating a Confidence Estimation Network (CENet), AcfNet adaptively adjusts the sharpness of the target distribution based on pixel-level uncertainty. This approach significantly improves precision and generalization, as evidenced by top-tier performance on the KITTI 2012 and 2015 benchmarks.

Introduction

Stereo matching is a fundamental computer vision task that estimates depth by finding horizontal shifts (disparity) between rectified image pairs. Modern architectures, such as PSMNet, construct a 3D cost volume and apply a "soft argmin" operation to regress the final disparity. A critical limitation of this approach is that the cost volume itself is not directly supervised. Consequently, the network may produce a messy internal state that happens to average out to the correct value during training but fails to generalize to challenging real-world scenarios like occlusions or textureless regions.

Methodology

AcfNet addresses the lack of structure in the cost volume through direct supervision and adaptive filtering.

Adaptive Unimodal Cost Volume Filtering

The core of AcfNet is the Adaptive Unimodal Cost Volume Filtering Module. Instead of relying solely on $L_1$ loss for the final disparity, the model defines a target unimodal probability distribution for the cost volume, centered at the ground truth disparity.

Overview of the AcfNet architecture, highlighting the Confidence Estimation Network (CENet) and the Adaptive Unimodal Cost Volume Filtering Module.

Confidence Estimation and Adaptive Variance

Not all pixels can be matched with equal certainty. AcfNet utilizes a Confidence Estimation Network (CENet) to produce a per-pixel confidence map. This map determines the variance ($\sigma$) of the target unimodal distribution:

  • High Confidence: For distinct features, the target distribution is sharp (low $\sigma$), forcing the model to be precise.
  • Low Confidence: In occluded or ambiguous regions, the distribution is flattened (high $\sigma$), allowing the model to express uncertainty rather than overfitting to noise.

Stereo Focal Loss

To handle the extreme class imbalance inherent in cost volumes—where only one disparity is correct among hundreds of candidates—the authors adapted the Focal Loss. This "Stereo Focal Loss" focuses training on hard-to-match pixels, further stabilizing the learning process.

Experiments and Results

The model was evaluated on the synthetic Scene Flow dataset and the real-world KITTI 2012 & 2015 benchmarks.

Quantitative Performance

AcfNet demonstrated superior accuracy over the baseline PSMNet. On the Scene Flow dataset, it reduced the End-Point Error (EPE) from 1.101 to 0.867. At the time of publication, AcfNet achieved 1st place on the KITTI 2012 leaderboard and 4th place on KITTI 2015.

Qualitative Analysis

Visual results indicate that AcfNet is particularly effective at preserving thin structures and sharp object boundaries, which are often blurred by traditional regression-based methods.

Qualitative comparison showing the left/right images, ground truth, AcfNet prediction, error map, and the predicted confidence map.

Conclusion

By shifting from indirect regression to direct, adaptive cost volume supervision, AcfNet provides a more robust framework for stereo matching. The use of a unimodal constraint, modulated by pixel-wise confidence, allows the network to learn a more physically meaningful internal representation, leading to state-of-the-art results and improved generalization to real-world driving environments.

Summary updated

BibTeX
@inproceedings{zhang2020adaptive,
  author = {Youmin Zhang and Yimin Chen and Xiao Bai and Jun Zhou and Kun Yu and Zhiwei Li and Kuiyuan Yang},
  title = {{Adaptive Unimodal Cost Volume Filtering for Deep Stereo Matching}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i07.6991},
}
Figure