Abstract
Unsupervised stereo matching offers a scalable alternative to supervised methods by eliminating the need for expensive ground-truth disparity data. However, these methods typically struggle with occlusions, where photometric consistency fails. We present CoT-Stereo, a novel framework that employs a "co-teaching" strategy between two deep neural networks. By interactively exchanging occlusion maps and utilizing a dynamic thresholding mechanism, CoT-Stereo effectively filters out noisy training signals in occluded regions. Our approach achieves state-of-the-art performance on the KITTI benchmarks, significantly narrowing the gap between unsupervised and supervised stereo matching while maintaining real-time inference speeds.
Introduction
Stereo matching is a fundamental task in computer vision, essential for depth perception in autonomous driving and robotics. While supervised deep learning models have achieved remarkable accuracy, their reliance on massive amounts of high-quality ground-truth data limits their applicability in diverse real-world environments.
Unsupervised methods circumvent this by optimizing for photometric consistency between stereo pairs. However, a major bottleneck remains: occlusions. In regions visible to only one camera, the photometric loss becomes unreliable, providing "noisy" gradients that lead to unstable training and inaccurate depth estimations. Standard networks often memorize these errors, degrading overall performance. CoT-Stereo addresses this by introducing a collaborative learning paradigm designed to identify and ignore these misleading signals.
Methodology: The Co-Teaching Framework
The core of CoT-Stereo is a dual-network architecture that leverages the "memorization effect" of deep learning—the tendency of networks to learn clean, simple patterns before fitting to noisy outliers.
Dual-Network Architecture and Occlusion Swapping
Instead of a single model, CoT-Stereo trains two identical networks (Network A and Network B) with different initializations. During training, the networks perform "peer review": Network A estimates an occlusion map and provides it to Network B to mask its loss calculation, and vice versa. Since the networks are initialized differently, they develop different error profiles. Swapping occlusion maps allows one network to act as a filter for the other's noise, preventing the propagation of inaccurate occlusion estimations.
Dynamic Thresholding and Hybrid Loss
To further refine the learning process, we implement a Dynamic Thresholding ($R(T)$) mechanism. The threshold starts high, allowing most pixels to contribute to the initial learning of basic structures. As training progresses, the threshold gradually decreases, filtering out pixels with high occlusion probability. This creates a curriculum-style learning environment where the model masters "easy" regions before attempting to handle complex occlusions. The final objective is a hybrid loss function combining photometric consistency (SSIM + L1), smoothness constraints, and data-augmentation losses.
Experimental Results
Benchmarking and Performance
We evaluated CoT-Stereo on the Scene Flow, KITTI Stereo 2012, and KITTI Stereo 2015 benchmarks. On the challenging KITTI 2015 dataset, CoT-Stereo achieved an F1-All score of 2.68%, establishing a new state-of-the-art for unsupervised methods. This performance significantly outperforms previous leaders such as SsSMnet (3.40%) and OASM-Net (8.98%).
Efficiency and Speed
Beyond accuracy, CoT-Stereo is designed for practical deployment. By utilizing the efficient LEAStereo backbone, the model maintains an inference time of just 0.30 seconds. As shown in the performance comparison below, CoT-Stereo occupies a unique position in the design space, offering accuracy comparable to some supervised models while remaining significantly faster than most unsupervised competitors.
Conclusion
CoT-Stereo demonstrates that a co-teaching strategy can effectively overcome the noise inherent in unsupervised stereo matching. By treating occlusion handling as a collaborative filtering task, we achieve a significant leap in both accuracy and robustness. While the dual-network training increases computational overhead during the learning phase, the resulting single-network inference remains highly efficient. Future work will explore the application of this co-teaching paradigm to other unsupervised vision tasks, such as optical flow and monocular depth estimation.
Summary updated