Toward dynamic recalibration and three-dimensional reconstruction in a structured light system

Youfu Li, B. ZhangJournal of the Optical Society of America A 2007 · DOI

TL;DR

This paper introduces a novel method for real-time recalibration and 3D reconstruction in structured light systems. It allows for dynamic adjustments of camera parameters on-the-fly using a single image, enabling applications with moving cameras or changing environments without the need for tedious offline recalibration.

What Problem Does This Paper Address?

Structured light systems are powerful tools for 3D reconstruction, but they typically require precise, offline calibration. This calibration is sensitive to changes in camera position, focus, or zoom, making it impractical for dynamic environments or when using handheld cameras. Existing methods often involve complex optimization or require multiple images, which can be slow and cumbersome. The paper addresses the need for a system that can adapt to changing conditions and perform 3D reconstruction in real-time with minimal user intervention, using only a single captured image.

Methods

The core of the proposed method lies in a dynamic recalibration approach that leverages a known scene plane. The system assumes that the light planes projected by the digital light projector have been pre-calibrated. The key innovation is the ability to determine camera parameters such as focal length, aspect ratio, and motion parameters (rotation and translation) online. This is achieved by analyzing the homography between the known scene plane and its projection in the image.

The method first establishes a relationship between 3D points on a plane and their 2D image projections using a homography matrix. By assuming a known scene plane and extracting feature points from a single image, the scene plane-to-image homography can be computed. This homography is then used to derive equations that allow for the direct, closed-form calculation of the camera's intrinsic parameters (focal length and aspect ratio) and extrinsic parameters (rotation and translation). Once these camera parameters are determined, the image-to-world transformation matrix can be computed, enabling 3D reconstruction. The paper also proposes a transformation-based method for 3D reconstruction, which is more efficient than traditional triangulation.

Datasets

Not discussed in the paper. The experiments were conducted using computer simulations with defined parameters and real-world data captured from a structured light system.

Key Insights and Ideas

  • Dynamic Recalibration with a Single Image: The most significant insight is the ability to perform camera recalibration and 3D reconstruction from a single image, eliminating the need for multiple views or complex iterative optimization.
  • Leveraging a Known Scene Plane: The method cleverly uses a simple, known scene plane (like a desktop or floor) as a reference for recalibration, avoiding the need for specialized calibration targets.
  • Closed-Form Solution: The derivation of closed-form solutions for camera parameters avoids the convergence issues and computational cost associated with iterative optimization methods.
  • Handheld Camera Compatibility: The dynamic recalibration and single-image requirement make the system suitable for handheld cameras, opening up new application possibilities.
  • Efficiency of Transformation-Based Reconstruction: The paper highlights the efficiency of a transformation-based 3D reconstruction method over traditional triangulation.

Key Results

  • Simulation Results: Computer simulations demonstrated that the method can accurately recover camera parameters and perform 3D reconstruction. The accuracy of the results improved significantly with an increasing number of feature points used for homography estimation, with around 40 points yielding acceptable results. The method also showed robustness to Gaussian noise in image pixels and small deviations in the principal point.
  • Real-World Experiments: Experiments with an industrial workpiece and a fan model showed acceptable reconstruction accuracy. The method was also tested in a dynamic scenario where the camera pose was changed while reconstructing two parallel planes, demonstrating consistent distance measurements with a low standard deviation.

Limitations and Future Work

  • Principal Point Assumption: The method assumes the principal point of the camera is fixed. While simulations showed some tolerance to small deviations, significant changes might impact accuracy.
  • Known Scene Plane Requirement: The method relies on the presence of a known scene plane. While this is a relaxed requirement compared to some methods, it is still a prerequisite.
  • Future Work: The authors suggest that the method's features make it particularly useful for applications like mobile robot navigation and desktop vision systems. Further exploration of its performance in more complex dynamic environments could be a future direction.

Summary updated

Abstract

We present a method for dynamic recalibration and 3D reconstruction via a structured light system. Assuming that the light planes cast from the digital light projector have been calibrated off-line, we show that the focal length, aspect ratio, and all motion parameters of the camera can be determined on-line. Then the 3D reconstruction can be carried out by either a traditional triangulation method or a more efficient transformation-based method. In the latter method, a single image is sufficient for the whole process of calibration and reconstruction. Thus a hand-held camera can be used. Computer simulation and real data experiments were carried out to validate the method.
BibTeX
@article{li2007toward,
  author = {Youfu Li and B. Zhang},
  title = {{Toward dynamic recalibration and three-dimensional reconstruction in a structured light system}},
  journal = {Journal of the Optical Society of America A},
  year = {2007},
  doi = {10.1364/josaa.24.000785},
}
Figure