Full Matching on Low Resolution for Disparity Estimation

Hong Zhang, Shenglun Chen, Zhihui Wang, Haojie Li, Wanli OuyangarXiv 2020 · arXiv

Abstract

Stereo disparity estimation is a critical task in computer vision, particularly for autonomous driving. While modern deep learning models utilize 4D cost volumes at reduced resolutions to manage computational costs, recovering high-resolution disparity maps remains a challenge. Traditional methods like linear interpolation often lead to blurry depth boundaries. This paper introduces Multistage Full Matching (MFM), a novel framework that decomposes the complex task of high-resolution similarity estimation into manageable stages. By decoupling similarity features and implementing a "Stages Mutual Aid" (SMA) strategy, MFM achieves state-of-the-art accuracy on major benchmarks, including Scene Flow and KITTI.

Introduction

Stereo matching involves finding the horizontal shift (disparity) of pixels between left and right image views to infer depth. To handle high-resolution imagery efficiently, most architectures construct a 4D cost volume at a lower resolution. However, the transition back to high-resolution disparity maps is often problematic. Linear interpolation assumes a smoothness in similarity scores that does not exist at object boundaries, while transposed convolutions suffer from "internal competition," where a single low-resolution feature must simultaneously predict multiple high-resolution scores.

Methodology

The proposed Multistage Full Matching (MFM) framework addresses these limitations through a "divide and conquer" strategy.

Overview of the MFM framework, illustrating the multistage 3D cost volume estimation and Stages Mutual Aid module.

Task Decomposition and Feature Decoupling

MFM divides the matching range into $k$ cells and estimates similarity scores across $n$ distinct stages. This decoupling allows the network to specialize its weights for specific subsets of candidates, alleviating the bottleneck where one feature maps to many disparate points.

Stages Mutual Aid (SMA)

To prevent earlier stages from propagating errors, the SMA mechanism allows for collaborative learning. It merges outputs from previous stages to provide a reference distribution for the current stage. This "voting" mechanism acts as a regularizer, ensuring consistency across the pipeline.

Sub-pixel Refinement

The final high-resolution 3D cost volume undergoes sub-pixel refinement using a modified parabolic fitting method, ensuring high precision in the resulting disparity map.

Experiments and Results

The MFM model was evaluated on the Scene Flow, KITTI 2012, and KITTI 2015 benchmarks.

Quantitative Performance

On the Scene Flow dataset, MFM achieved an End-Point Error (EPE) of 0.66, significantly outperforming GwcNet (0.77). On KITTI 2015, it reduced the D1-all error to 1.87%, surpassing competitive models like AcfNet (1.89%). Ablation studies confirmed that the SMA module alone improved the EPE from 0.74 to 0.66.

Qualitative Analysis

Qualitative results demonstrate that MFM produces sharper depth boundaries and fewer artifacts compared to existing state-of-the-art methods.

Qualitative comparison of disparity maps between MFM (Ours), AcfNet, and AANet+. MFM shows superior detail in complex regions.

Conclusion

The MFM framework demonstrates that high-resolution matching information can be effectively recovered from low-resolution features by treating the task as a learned, multistage decoupling process. By avoiding the pitfalls of linear interpolation and internal competition, MFM sets a new standard for accuracy in stereo disparity estimation. Future work may extend this multistage logic to other volumetric tasks such as optical flow or 3D object detection.

Summary updated

Abstract

A Multistage Full Matching disparity estimation scheme (MFM) is proposed in this work. We demonstrate that decouple all similarity scores directly from the low-resolution 4D volume step by step instead of estimating low-resolution 3D cost volume through focusing on optimizing the low-resolution 4D volume iteratively leads to more accurate disparity. To this end, we first propose to decompose the full matching task into multiple stages of the cost aggregation module. Specifically, we decompose the high-resolution predicted results into multiple groups, and every stage of the newly designed cost aggregation module learns only to estimate the results for a group of points. This alleviates the problem of feature internal competitive when learning similarity scores of all candidates from one low-resolution 4D volume output from one stage. Then, we propose the strategy of \emph{Stages Mutual Aid}, which takes advantage of the relationship of multiple stages to boost similarity scores estimation of each stage, to solve the unbalanced prediction of multiple stages caused by serial multistage framework. Experiment results demonstrate that the proposed method achieves more accurate disparity estimation results and outperforms state-of-the-art methods on Scene Flow, KITTI 2012 and KITTI 2015 datasets.
BibTeX
@misc{zhang2020full,
  author = {Hong Zhang and Shenglun Chen and Zhihui Wang and Haojie Li and Wanli Ouyang},
  title = {{Full Matching on Low Resolution for Disparity Estimation}},
  year = {2020},
  eprint = {2012.05586},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2012.05586},
}
Figure