STEREO IMAGE DENSE MATCHING BY INTEGRATING SIFT AND SGM ALGORITHM

Yu Zhou, Yu Song, Kevin Lü2018 · DOI

Abstract

Semi-global matching (SGM) performs dynamic programming by treating different path directions equally, failing to consider the impact of image structure on cost aggregation. Furthermore, as the disparity search range expands, algorithm accuracy and efficiency decrease. This paper presents a dense matching algorithm integrating SIFT and SGM. SIFT-matched pairs serve as control points to direct dynamic programming and truncate error propagation. Additionally, matching accuracy is improved by using the gradient direction of detected feature points to modify path weights. Experimental results on Middlebury and CE-3 lunar datasets demonstrate that the proposed algorithm effectively cuts off error propagation, reduces the search range, and improves matching accuracy.

Introduction

Stereo matching remains a challenging task in 3D reconstruction due to shadows, occlusions, and repetitive textures. Semi-Global Matching (SGM) is widely used for its balance between speed and accuracy. However, SGM suffers from three primary limitations:

  1. Equal Weighting: It treats all eight path directions (horizontal, vertical, diagonal) equally during cost aggregation, ignoring image texture.
  2. Error Propagation: Mismatches can propagate along dynamic programming paths, creating artifacts.
  3. Computational Load: Memory requirements grow exponentially with the disparity search range.
The eight path directions used in standard Semi-Global Matching (SGM) cost aggregation.

Methodology

The proposed approach modifies the standard SGM algorithm by incorporating SIFT features at multiple stages of the pipeline.

SIFT-Guided Control Points

The algorithm first identifies sparse, reliable matching pairs using SIFT. These pairs act as "control points" or anchors. During the SGM phase, these points serve as barriers that truncate error propagation. When dynamic programming encounters a control point, it is constrained to the known correct value, preventing mismatches from affecting subsequent pixels.

Adaptive Search Range and Path Weighting

By combining image segmentation with SIFT anchors, the algorithm calculates a localized search range, reducing memory usage and false matches. Furthermore, instead of treating all eight aggregation directions equally, the method utilizes the gradient direction and amplitude of feature points. Lower weights are assigned to paths aligning with high-gradient directions, making the aggregation process aware of image structure and preserving edges.

Flowchart of the integrated SIFT and SGM dense matching algorithm.

Experimental Results

The algorithm was evaluated using the Middlebury Stereo Datasets (Aloe and Cones) and Chang’e 3 (CE-3) lunar data.

Quantitative Performance

Using the Percentage of Bad Matching Pixels (PBM) metric, the modified algorithm showed measurable improvements. On the Cones dataset, accuracy increased from 78.33% to 81.41%. On the Aloe dataset, it rose from 75.60% to 76.83%.

Visual Quality and Lunar Data

In the CE-3 lunar images, characterized by weak textures and harsh illumination, the SIFT-integrated SGM produced significantly cleaner disparity maps. The SIFT matching phase achieved a Root Mean Squared Error (RMSE) of 0.8189 pixels, providing a robust foundation for dense matching.

Conclusion

Integrating SIFT features as control points and using gradient information for path weighting significantly enhances SGM. This hybrid approach effectively mitigates error propagation and improves efficiency by narrowing the disparity search range. Future work will focus on enhancing SIFT performance in ultra-low-texture environments and further refining dynamic path weighting.

Summary updated

Abstract

Semi-global matching(SGM) performs the dynamic programming by treating the different path directions equally. It does not consider the impact of different path directions on cost aggregation, and with the expansion of the disparity search range, the accuracy and efficiency of the algorithm drastically decrease. This paper presents a dense matching algorithm by integrating SIFT and SGM. It takes the successful matching pairs matched by SIFT as control points to direct the path in dynamic programming with truncating error propagation. Besides, matching accuracy can be improved by using the gradient direction of the detected feature points to modify the weights of the paths in different directions. The experimental results based on Middlebury stereo data sets and CE-3 lunar data sets demonstrate that the proposed algorithm can effectively cut off the error propagation, reduce disparity search range and improve matching accuracy.
BibTeX
@inproceedings{zhou2018stereo,
  author = {Yu Zhou and Yu Song and Kevin Lü},
  title = {{STEREO IMAGE DENSE MATCHING BY INTEGRATING SIFT AND SGM ALGORITHM}},
  booktitle = {˜The},
  year = {2018},
  doi = {10.5194/isprs-archives-xlii-3-2565-2018},
}
Figure