CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching

Zhelun Shen, Yuchao Dai, Zhibo Rao

Abstract

Stereo matching has seen significant progress with deep learning, yet models often struggle with generalization across diverse environments. CFNet (Cascade and Fused cost volume Network) addresses this "brittleness" by introducing a robust architecture that performs reliably without dataset-specific tuning. By fusing multiple low-resolution cost volumes and utilizing a variance-based uncertainty measure to adaptively prune the search space, CFNet achieves state-of-the-art cross-domain generalization. It notably secured 1st place in the Robust Vision Challenge 2020.

Introduction

While deep learning has advanced stereo matching—the process of estimating depth from image pairs—existing models frequently suffer from poor generalization. A model trained on sunny driving scenes (e.g., KITTI) often fails in indoor environments (e.g., Middlebury) or grayscale industrial settings (e.g., ETH3D). This failure stems from two primary factors: domain shifts in lighting and texture, and unbalanced disparity distributions across different datasets. CFNet is designed to overcome these challenges by focusing on structural consistency and adaptive search ranges.

Methodology

CFNet employs a three-part architecture to ensure robustness:

  1. Pyramid Feature Extraction: A Siamese UNet-like structure extracts multi-scale features, providing a rich representation of the input images.
  2. Fused Cost Volume: To mitigate domain shifts, CFNet fuses multiple low-resolution dense cost volumes. Unlike previous methods that discarded low-resolution data as blurry, CFNet leverages them to capture broad structural geometry, which remains stable across different lighting and textures.
  3. Cascade Cost Volume with Adaptive Uncertainty: This core innovation uses a coarse-to-fine approach. At each stage, the model calculates uncertainty based on the variance of the probability distribution. High confidence (low uncertainty) leads to a narrower search space, while low confidence (high uncertainty) expands it. This allows the model to adaptively prune unlikely matches while maintaining flexibility in difficult regions.

Experimental Evaluation

The authors evaluated CFNet on several rigorous benchmarks: SceneFlow (synthetic pre-training), KITTI 2012 & 2015 (urban driving), Middlebury (indoor), and ETH3D (grayscale). Crucially, robustness was tested by training on a mix of datasets and evaluating with fixed model parameters across all domains.

Qualitative comparison of disparity maps generated by CFNet, GANet, and HSMNet across KITTI, Middlebury, and ETH3D datasets.

Key Results and Discussion

CFNet demonstrated exceptional performance across all tested domains, outperforming specialized models that often failed when tested outside their training distribution.

  • Robust Vision Challenge 2020: CFNet won 1st place in the stereo task, confirming its robustness.
  • Quantitative Performance: As shown in the benchmarking results, CFNet maintains low error rates across disparate datasets like Middlebury, ETH3D, and KITTI, whereas models like GANet or HSMNet show significant performance drops in non-target domains.
  • Efficiency: CFNet achieves an inference time of under 200ms on KITTI images, making it suitable for near-real-time applications in robotics and autonomous driving.
  • Accuracy: On KITTI 2015, it achieved a 1.88% D1_all error rate, a notable improvement over its base architecture, CasStereo.
Quantitative comparison of error rates for CFNet and other state-of-the-art models across Middlebury, ETH3D, and KITTI benchmarks.

Conclusion

CFNet proves that generalization does not require sacrificing top-tier performance. By prioritizing global structural features and adaptive uncertainty, it creates an "all-terrain" model for depth estimation. Future work includes extending this architecture to semi-supervised or self-supervised learning frameworks to further enhance its ability to learn from unlabeled real-world data.

Summary updated

BibTeX
@inproceedings{shen2021cfnet,
  author = {Zhelun Shen and Yuchao Dai and Zhibo Rao},
  title = {{CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2021},
  doi = {10.1109/cvpr46437.2021.01369},
}
Figure