ELFNet: Evidential Local-global Fusion for Stereo Matching

Jieming Lou, Weide Liu, Zhuo Chen, Fayao Liu, Jun Cheng

Abstract

Stereo matching is a cornerstone of autonomous systems, yet deep learning models often struggle with overconfidence and the trade-off between local texture and global context. We introduce ELFNet, a framework that treats stereo matching as an evidential regression problem. By predicting the parameters of a Normal-Inverse Gamma (NIG) distribution, ELFNet explicitly quantifies aleatoric and epistemic uncertainty. This uncertainty is leveraged to intelligently fuse local-focused convolutional features with global-focused transformer features. ELFNet achieves state-of-the-art accuracy on the Scene Flow dataset and demonstrates superior zero-shot generalization on real-world benchmarks like KITTI and Middlebury.

Introduction

Accurate depth estimation from stereo images is vital for safety-critical applications like autonomous driving. Traditional deep learning models provide point estimates of disparity without quantifying uncertainty, which can lead to overconfident and potentially dangerous errors. Furthermore, there is a persistent challenge in balancing the fine-grained local details captured by Convolutional Neural Networks (CNNs) with the long-range dependencies captured by Transformers. ELFNet addresses these issues by using evidential learning to guide a robust fusion of local and global information.

Methodology: Evidential Local-global Fusion

The core innovation of ELFNet is the Evidential Local-global Fusion (ELF) framework. Instead of predicting a single disparity value, the model outputs the parameters of a Normal-Inverse Gamma (NIG) distribution, allowing for the calculation of both aleatoric (data-inherent) and epistemic (model-knowledge) uncertainty.

The ELFNet architecture utilizes a dual-branch approach, fusing CNN-based local features and Transformer-based global features through intra- and inter-evidential fusion modules.

Two-Stage Fusion Strategy

As illustrated in the architecture, ELFNet employs a hierarchical fusion process:

  1. Intra-evidential Fusion: Within the CNN branch, multi-scale cost volumes are merged using the mathematical properties of the NIG distribution. This allows the model to combine evidence from different spatial resolutions effectively.
  2. Inter-evidential Fusion: This stage acts as the bridge between the CNN (PCWNet) and Transformer (STTR) branches. By weighting the predictions based on their respective confidence levels, the framework dynamically selects the most reliable information for each pixel.

Experimental Results

ELFNet was rigorously tested on industry-standard benchmarks, including Scene Flow, KITTI 2012/2015, and Middlebury 2014.

Quantitative Performance

  • State-of-the-Art Accuracy: On the Scene Flow dataset, ELFNet achieved an End Point Error (EPE) of 0.33px, a 19.5% improvement over the previous leader, CSTR.
  • Cross-Domain Robustness: When trained on synthetic data and tested on the real-world Middlebury dataset, ELFNet showed a 17.5% improvement in EPE over the PCWNet baseline, proving its superior ability to generalize to unfamiliar environments.

Qualitative Analysis

Visualization of the uncertainty maps demonstrates that ELFNet correctly identifies high-risk areas, such as object boundaries and occluded regions, providing a critical safety buffer for downstream tasks.

Qualitative comparison of disparity and uncertainty maps. ELFNet (b) shows improved detail and reliability compared to baseline methods in complex scenes.

Conclusion

ELFNet demonstrates that uncertainty quantification is not merely a diagnostic tool but a powerful mechanism for architectural fusion. By integrating evidential learning, ELFNet achieves a new standard for accuracy and reliability in stereo matching. While the current dual-branch architecture presents a challenge for real-time inference, future work will explore more efficient structures, such as sparse cost volumes, to maintain these benefits in high-speed applications.

Summary updated

Abstract

Although existing stereo matching models have achieved continuous improvement, they often face issues related to trustworthiness due to the absence of uncertainty estimation. Additionally, effectively leveraging multi-scale and multi-view knowledge of stereo pairs remains unexplored. In this paper, we introduce the \textbf{E}vidential \textbf{L}ocal-global \textbf{F}usion (ELF) framework for stereo matching, which endows both uncertainty estimation and confidence-aware fusion with trustworthy heads. Instead of predicting the disparity map alone, our model estimates an evidential-based disparity considering both aleatoric and epistemic uncertainties. With the normal inverse-Gamma distribution as a bridge, the proposed framework realizes intra evidential fusion of multi-level predictions and inter evidential fusion between cost-volume-based and transformer-based stereo matching. Extensive experimental results show that the proposed framework exploits multi-view information effectively and achieves state-of-the-art overall performance both on accuracy and cross-domain generalization. The codes are available at https://github.com/jimmy19991222/ELFNet.
BibTeX
@inproceedings{lou2023elfnet,
  author = {Jieming Lou and Weide Liu and Zhuo Chen and Fayao Liu and Jun Cheng},
  title = {{ELFNet: Evidential Local-global Fusion for Stereo Matching}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
}
Figure