Vision Measurement Method Based on Plate Glass Window Refraction Model in Tunnel Construction

Zhen Wu, Junzhou Huo, Haidong Zhang, Fan Yang, Shangqi Chen, Zhihao FengSensors 2024 · DOI

TL;DR

This paper tackles the challenge of accurate 3D vision measurement in harsh tunnel construction environments where cameras are protected by glass windows. They introduce a novel method that models and corrects for the refractive effects of this glass, significantly improving measurement accuracy compared to traditional approaches.

What Problem Does This Paper Address?

Tunnel Boring Machines (TBMs) operate in challenging environments with high humidity and dust, necessitating protective glass covers for cameras. This glass, however, introduces refraction, which distorts images and invalidates standard pinhole camera models. This distortion leads to significant errors in depth and positioning measurements, crucial for tasks like robotic tool changing in TBMs. Existing methods often struggle with non-parallel camera-to-glass orientations or are overly complex. This paper aims to provide a comprehensive and accurate solution for vision measurement under such refractive conditions.

Methods

The authors propose a multi-stage approach:

  • Non-Parallel Plane Refraction Imaging Model: They establish a new imaging model that accounts for the camera's optical axis not being parallel to the refractive plane. This involves using a "dynamic virtual focal length" and the Rodriguez formula for camera rotation correction.
  • Independent Refractive Plane Epipolar Constraint Model: Refraction breaks the standard epipolar constraint used in stereo vision. This paper introduces a new model that handles independent refractive planes, allowing for accurate feature point matching even when the camera's optical axis is not aligned with the glass normal.
  • Independent Refractive Plane Triangulation Model: Building on the previous steps, they develop a triangulation model that can accurately reconstruct 3D points from stereo images taken through refractive glass. This model addresses the failure of traditional triangulation methods under these conditions.

Datasets

Not discussed in the paper. The experiments used custom-built experimental platforms and targets.

Key Insights and Ideas

  • Glass Refraction is a Major Hurdle: The paper clearly demonstrates that even seemingly simple glass plates introduce significant distortions that traditional vision models cannot handle.
  • Non-Parallelism is Key: The misalignment between the camera's optical axis and the glass's normal vector is a critical factor that existing models often simplify or ignore, leading to errors.
  • Epipolar Constraint Failure: Refraction fundamentally breaks the geometric relationships that underpin stereo vision, requiring a new approach to feature matching.
  • Unified Solution: The paper offers a cohesive framework that addresses image distortion, camera rotation, epipolar constraints, and triangulation, all within the context of refractive glass.

Key Results

The proposed method significantly improves measurement accuracy:

  • Positioning Error: RMSE reduced from 1.3441 mm and 1.9937 mm (before correction) to 0.1135 mm and 0.1484 mm (after correction) for monocular measurements.
  • Measurement Error: RMSE reduced from 2.3821 mm and 2.4037 mm (before correction) to 0.2130 mm and 0.2136 mm (after correction) for line segments.
  • Depth of Field Error: RMSE reduced from 2.9902 mm to 0.3187 mm.
  • 3D Positioning Error: RMSE reduced from 3.5661 mm to 0.3465 mm.

These results show a substantial improvement, often by an order of magnitude, in accuracy.

Limitations and Future Work

  • Simulated vs. Real Environments: While experiments were conducted under simulated tunnel conditions (polluted glass), real-world tunnel environments can be even harsher.
  • Glass Properties: The experiments used K9 optical glass. Future work should investigate the performance with explosion-proof tempered glass, which may have different optical properties.
  • Image Restoration: For severely degraded images due to extreme pollution or obstruction, image restoration techniques (e.g., deep learning) could be explored.
  • Generalization: While the method is designed for tunnel construction, its applicability to other vision systems with glass protection (e.g., underwater robotics) could be further explored.

Summary updated

BibTeX
@article{wu2024vision,
  author = {Zhen Wu and Junzhou Huo and Haidong Zhang and Fan Yang and Shangqi Chen and Zhihao Feng},
  title = {{Vision Measurement Method Based on Plate Glass Window Refraction Model in Tunnel Construction}},
  journal = {Sensors},
  year = {2024},
  doi = {10.3390/s24010066},
}
Figure