Abstract
Stereo matching is a fundamental task in computer vision, yet traditional deep learning models often struggle with reliable uncertainty estimation. This paper presents a novel evidential learning framework that replaces the standard single-Gaussian assumption with a Mixture-of-Gaussian (MoG) distribution. By modeling pixels through multiple potential distributions and utilizing an Expectation-Maximization (EM) approach, the proposed method achieves state-of-the-art depth estimation and robust uncertainty quantification, particularly in complex cross-domain scenarios.
Introduction
Stereo matching—the process of calculating depth by comparing two images—is critical for autonomous systems. However, existing models frequently fail to "know what they don't know." Traditional Evidential Deep Learning (EDL) attempts to address this by placing priors on model predictions, but typically assumes a single Gaussian distribution for every pixel. In reality, image data is often multimodal; for instance, a pixel near an object boundary may represent foreground, background, or noise. Forcing these complex realities into a single "bell curve" results in imprecise depth maps and unreliable uncertainty estimates, which can be hazardous in safety-critical applications.
Methodology: Mixture-of-Gaussian Evidential Learning (MOG EL)
To overcome the limitations of single-Gaussian models, the authors propose the Mixture-of-Gaussian based Evidential Learning (MOG EL) framework. Instead of a single prediction, the network estimates parameters for multiple Gaussian components that could explain the observed data.
Key Technical Components
- Probabilistic Hierarchy: The framework employs a Normal-Inverse-Gamma (NIG) distribution as a higher-order prior for each component in the mixture. This enables the separate estimation of aleatoric uncertainty (inherent data noise) and epistemic uncertainty (model-related uncertainty).
- Expectation-Maximization (EM) Training: To handle the non-convex optimization challenges inherent in MoG models, an EM-style approach is integrated directly into the neural network's training loop.
- Architecture: Built upon the STTR (Stereo Transformer) backbone, the model is augmented with five specialized heads to predict the necessary hyperparameters: $\gamma, \nu, \alpha, \beta, \pi$.
- Evidence Penalty: An "incorrect evidence penalty" is applied during training to mitigate overconfidence in erroneous predictions.
Experimental Setup and Datasets
The model's performance was rigorously evaluated across several industry-standard benchmarks:
- Scene Flow (FlyingThings 3D): A large-scale synthetic dataset for primary training and in-domain validation.
- KITTI 2015: Real-world driving sequences used to assess generalization in high-stakes outdoor environments.
- Middlebury 2014: High-resolution indoor stereo pairs for cross-domain evaluation.
Key Insights: The Power of Multimodality
The transition to a multimodal approach provides several distinct advantages. By allowing for $K$ Gaussian components (with $K=20$ found to be optimal), the model can effectively segment objects in depth space, leading to significantly sharper object edges.
Furthermore, the model demonstrates exceptional zero-shot generalization. Because it is "aware" of its own uncertainty, it avoids overfitting on synthetic training data, resulting in superior performance on unseen real-world datasets. The framework also exhibits enhanced adversarial robustness; under attacks like FGSM, the model flags perturbed areas with high uncertainty rather than providing confident, incorrect answers.
Results and Performance
The MOG EL framework established new state-of-the-art (SOTA) benchmarks:
- Accuracy: On Scene Flow, it achieved an End-Point Error (EPE) of 0.31, a 25% improvement over the previous SOTA (ELFNet).
- Cross-Domain Performance: When transferred from synthetic data to the real-world Middlebury dataset, EPE was reduced from 2.33 to 1.30, a 44.2% improvement.
- Efficiency: The computational overhead introduced by the MoG heads is marginal, making it a practical enhancement for existing architectures.
Conclusion and Future Directions
While MOG EL represents a significant leap forward, several avenues for future research remain. Currently, the number of Gaussian components ($K$) is a fixed hyperparameter; future iterations could utilize Dirichlet Process mixtures for dynamic component determination. Additionally, while cross-domain results are strong, the synthetic-to-real gap persists under extreme lighting or weather conditions. Finally, further optimization of the EM-based loss calculation could enhance performance in ultra-low-latency environments.
Summary updated