Co-Teaching: an Ark to Unsupervised Stereo Matching

Hengli Wang, Rui Fan, Ming Liu

Abstract

Unsupervised stereo matching offers a scalable alternative to supervised methods by eliminating the need for expensive ground-truth disparity data. However, these methods typically struggle with occlusions, where photometric consistency fails. We present CoT-Stereo, a novel framework that employs a "co-teaching" strategy between two deep neural networks. By interactively exchanging occlusion maps and utilizing a dynamic thresholding mechanism, CoT-Stereo effectively filters out noisy training signals in occluded regions. Our approach achieves state-of-the-art performance on the KITTI benchmarks, significantly narrowing the gap between unsupervised and supervised stereo matching while maintaining real-time inference speeds.

Introduction

Stereo matching is a fundamental task in computer vision, essential for depth perception in autonomous driving and robotics. While supervised deep learning models have achieved remarkable accuracy, their reliance on massive amounts of high-quality ground-truth data limits their applicability in diverse real-world environments.

Unsupervised methods circumvent this by optimizing for photometric consistency between stereo pairs. However, a major bottleneck remains: occlusions. In regions visible to only one camera, the photometric loss becomes unreliable, providing "noisy" gradients that lead to unstable training and inaccurate depth estimations. Standard networks often memorize these errors, degrading overall performance. CoT-Stereo addresses this by introducing a collaborative learning paradigm designed to identify and ignore these misleading signals.

Methodology: The Co-Teaching Framework

The core of CoT-Stereo is a dual-network architecture that leverages the "memorization effect" of deep learning—the tendency of networks to learn clean, simple patterns before fitting to noisy outliers.

Dual-Network Architecture and Occlusion Swapping

Instead of a single model, CoT-Stereo trains two identical networks (Network A and Network B) with different initializations. During training, the networks perform "peer review": Network A estimates an occlusion map and provides it to Network B to mask its loss calculation, and vice versa. Since the networks are initialized differently, they develop different error profiles. Swapping occlusion maps allows one network to act as a filter for the other's noise, preventing the propagation of inaccurate occlusion estimations.

The CoT-Stereo architecture illustrates the interaction between Network A and Network B, including the swapping of occlusion maps and shared weights strategy.

Dynamic Thresholding and Hybrid Loss

To further refine the learning process, we implement a Dynamic Thresholding ($R(T)$) mechanism. The threshold starts high, allowing most pixels to contribute to the initial learning of basic structures. As training progresses, the threshold gradually decreases, filtering out pixels with high occlusion probability. This creates a curriculum-style learning environment where the model masters "easy" regions before attempting to handle complex occlusions. The final objective is a hybrid loss function combining photometric consistency (SSIM + L1), smoothness constraints, and data-augmentation losses.

Experimental Results

Benchmarking and Performance

We evaluated CoT-Stereo on the Scene Flow, KITTI Stereo 2012, and KITTI Stereo 2015 benchmarks. On the challenging KITTI 2015 dataset, CoT-Stereo achieved an F1-All score of 2.68%, establishing a new state-of-the-art for unsupervised methods. This performance significantly outperforms previous leaders such as SsSMnet (3.40%) and OASM-Net (8.98%).

Efficiency and Speed

Beyond accuracy, CoT-Stereo is designed for practical deployment. By utilizing the efficient LEAStereo backbone, the model maintains an inference time of just 0.30 seconds. As shown in the performance comparison below, CoT-Stereo occupies a unique position in the design space, offering accuracy comparable to some supervised models while remaining significantly faster than most unsupervised competitors.

Comparison of F1-All error versus inference time on the KITTI 2015 benchmark. CoT-Stereo (red star) achieves the best balance of accuracy and speed among unsupervised approaches.

Conclusion

CoT-Stereo demonstrates that a co-teaching strategy can effectively overcome the noise inherent in unsupervised stereo matching. By treating occlusion handling as a collaborative filtering task, we achieve a significant leap in both accuracy and robustness. While the dual-network training increases computational overhead during the learning phase, the resulting single-network inference remains highly efficient. Future work will explore the application of this co-teaching paradigm to other unsupervised vision tasks, such as optical flow and monocular depth estimation.

Summary updated

Abstract

Stereo matching is a key component of autonomous driving perception. Recent unsupervised stereo matching approaches have received adequate attention due to their advantage of not requiring disparity ground truth. These approaches, however, perform poorly near occlusions. To overcome this drawback, in this paper, we propose CoT-Stereo, a novel unsupervised stereo matching approach. Specifically, we adopt a co-teaching framework where two networks interactively teach each other about the occlusions in an unsupervised fashion, which greatly improves the robustness of unsupervised stereo matching. Extensive experiments on the KITTI Stereo benchmarks demonstrate the superior performance of CoT-Stereo over all other state-of-the-art unsupervised stereo matching approaches in terms of both accuracy and speed. Our project webpage is https://sites.google.com/view/cot-stereo.
BibTeX
@inproceedings{wang2021co,
  author = {Hengli Wang and Rui Fan and Ming Liu},
  title = {{Co-Teaching: an Ark to Unsupervised Stereo Matching}},
  booktitle = {ICIP},
  year = {2021},
  url = {https://openreview.net/forum?id=L_eXALaP7_},
}
Figure