Geological Borehole Video Image Stitching Method Based on Local Homography Matrix Offset Optimization

Zhaopeng Deng, Song Shengzhi, Shuangyang Han, Zeqi Liu, Qiang Wang, Liuyang JiangSensors 2023 · DOI

TL;DR

This paper introduces a novel image stitching method for geological borehole videos, designed to overcome interference and create a comprehensive panoramic view of the borehole wall. By combining SURF for feature detection, MSAC for robust matching, and a new Local Homography Matrix Offset Optimization (LHOO) algorithm, the method significantly improves stitching accuracy and produces high-quality panoramas.

What Problem Does This Paper Address?

Geological exploration often relies on borehole cameras to inspect the inner walls of boreholes. However, the unique shooting environment and inherent characteristics of these videos (like limited texture and color variation) lead to significant interference and inaccurate image matching, making it difficult to create a clear, continuous panorama. This lack of accurate visualization hinders detailed analysis of rock mass structural planes, such as trends, cracks, and interfaces. Existing image stitching techniques struggle with these challenges, often resulting in mismatched points and poor stitching quality.

Methods

The proposed method tackles this problem through a multi-stage approach:

  • Feature Detection and Rough Matching: The Speeded-Up Robust Features (SURF) algorithm is employed to extract key feature points from the borehole video frames. These points are then used for an initial, rough matching between adjacent frames.
  • Robust Homography Estimation: The M-Estimator Sample Consensus (MSAC) algorithm is utilized to filter out mismatched feature point pairs and accurately estimate the homography matrix, which describes the geometric transformation between the images. MSAC is an improvement over RANSAC, offering better robustness to outliers and threshold sensitivity.
  • Local Homography Matrix Offset Optimization (LHOO): This is the core novel contribution. Recognizing that borehole videos often exhibit a larger vertical offset than horizontal, the LHOO algorithm specifically optimizes the vertical offset between consecutive frames. It achieves this by:
  • Obtaining multiple homography matrices for a group of frames.
  • Calculating a threshold for the vertical offset based on camera parameters and frame rate.
  • Filtering out homography matrices that suggest an offset exceeding this threshold.
  • Extracting the optimal vertical offset from the remaining candidates using a statistical optimization method (Pearson mode).
  • Cylindrical Panorama Generation: The optimized vertical offsets are then used sequentially to stitch the unfolded borehole images frame by frame, creating a complete cylindrical panorama.

Datasets

The paper does not explicitly mention training datasets. For evaluation, the authors used:

  • Natural scene images (for initial feature extraction comparison).
  • Geological borehole images captured by an Axial View Panoramic Borehole Camera System (APBCS).
  • A real-time stitching experiment on 225 consecutive frames from a borehole video.

The authors highlight that the geological borehole images present a unique challenge due to their simple color, minimal texture, and limited feature points, differentiating them from more complex natural scene images.

Key Insights and Ideas

  • The "Vertical Offset Problem": The paper correctly identifies that in borehole imaging, the camera's movement often results in a dominant vertical displacement, which is a critical factor for accurate stitching.
  • LHOO for Targeted Optimization: The LHOO algorithm's focus on optimizing the vertical offset is a clever adaptation to the specific challenges of borehole video. This targeted approach is more effective than general-purpose stitching methods.
  • SURF and MSAC Synergy: The combination of SURF for speed and robustness in feature detection, followed by MSAC for outlier rejection, provides a solid foundation for accurate homography estimation.
  • Practical Relevance: The method is directly applicable to real-world geological drilling scenarios, offering a practical solution for visualizing and analyzing borehole structures.

Key Results

  • Improved Matching Accuracy: The proposed method, using SURF and LHOO, demonstrated significantly improved matching accuracy compared to SIFT, Harris, and ORB algorithms on geological borehole images. The paper claims a 20-60% improvement in matching accuracy over these methods.
  • High-Quality Panorama: The final stitched panorama of 225 consecutive frames exhibited a good visual effect, clearly reflecting the borehole's inner wall texture and structural planes.
  • Real-Time Performance: The average stitching time per frame was 100 ms, which the authors state meets the requirements for real-time systems.

Limitations and Future Work

  • Algorithm Complexity: The paper acknowledges that the multiple offset statistical optimization in LHOO increases the algorithm's complexity, potentially making it less suitable for some extremely high-speed real-time splicing systems.
  • Stitching Seams: The authors note that image changes between frames can lead to stitching seams, which they plan to address in future work.
  • Future Improvements: Future research will focus on:
  • Further improving feature point extraction and matching.
  • Developing a more rapid and accurate statistical optimization method.
  • Effectively eliminating stitching seams to enhance panorama quality.

Summary updated

Abstract

Due to the influence of the shooting environment and inherent image characteristics, there is a large amount of interference in the process of image stitching a geological borehole video. To accurately match the acquired image sequences in the inner part of a borehole, this paper presents a new method of stitching an unfolded borehole image, which uses the image generated from the video to construct a large-scale panorama. Firstly, the speeded-up robust feathers (SURF) algorithm is used to extract the image feature points and complete the rough matching. Then, the M-estimator sample consensus (MSAC) algorithm is introduced to remove the mismatched point pairs and obtain the homography matrix. Subsequently, we propose a local homography matrix offset optimization (LHOO) algorithm to obtain the optimal offset. Finally, the above process is cycled frame by frame, and the image sequence is continuously stitched to complete the construction of a cylindrical borehole panorama. The experimental results show that compared with those of the SIFT, Harris, ORB and SURF algorithms, the matching accuracy of our algorithm has been greatly improved. The final test is carried out on 225 consecutive video frames, and the panorama has a good visual effect, and the average time of each frame is 100 ms, which basically meets the requirements of the project.
BibTeX
@inproceedings{deng2023geological,
  author = {Zhaopeng Deng and Song Shengzhi and Shuangyang Han and Zeqi Liu and Qiang Wang and Liuyang Jiang},
  title = {{Geological Borehole Video Image Stitching Method Based on Local Homography Matrix Offset Optimization}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23020632},
}
Figure