3D Modelling Using Geometric Constraints: A Parallelepiped Based Approach

Marta Wilczkowiak, Edmond Boyer, Peter F. SturmECCV 2002 · DOI

TL;DR

This paper introduces a novel approach for 3D modeling and camera calibration using parallelepipeds as geometric constraints. By exploiting the inherent geometric properties of these shapes, the method offers a more robust and efficient way to reconstruct scenes and calibrate cameras, even from a small number of images with minimal user interaction.

What Problem Does This Paper Address?

Reconstructing 3D scenes from 2D images and determining camera parameters (calibration) are fundamental problems in computer vision. Traditional methods often require many images, precise point correspondences, or specific camera motions. While using geometric constraints from man-made environments (like parallel lines or right angles) can simplify these tasks, existing approaches often struggle to fully exploit the rich information contained within higher-level primitives like cuboids or parallelepipeds. This paper aims to bridge this gap by proposing a unified framework that leverages the geometric properties of parallelepipeds for both camera calibration and 3D scene reconstruction, reducing the need for extensive user input and complex algorithms.

Methods

The core of the paper lies in the concept of duality between camera intrinsic parameters and parallelepiped intrinsic parameters. This duality is formalized using matrices that represent these intrinsic properties. The authors introduce the canonic projection matrix ($X$), which encapsulates the projection of a parallelepiped into an image. This matrix relates the camera's intrinsic parameters (represented by $w$, the image of the absolute conic) and the parallelepiped's intrinsic parameters (represented by $\mu$). The key equation is $X^T \omega X \sim \mu$, which establishes the duality.

The approach then extends to multiple views and multiple parallelepipeds. By exploiting this duality, the system can derive linear constraints on the unknown intrinsic parameters. The calibration process involves:

  1. Parameterization: Defining the problem in terms of a single set of intrinsic parameters (either for a camera or a parallelepiped).
  2. Using Prior Knowledge: Incorporating known geometric information about cameras (e.g., zero skew, known aspect ratio) and parallelepipeds (e.g., right angles, known length ratios) to derive linear equations.
  3. Solving Linear Systems: Using Singular Value Decomposition (SVD) to solve the system of linear equations for the intrinsic parameters. If the system is under-constrained, quadratic constraints are used to resolve ambiguities.
  4. Pose Estimation: Once intrinsics are known, the relative pose (rotation and translation) between cameras and parallelepipeds is estimated.
  5. 3D Reconstruction: Reconstructing the 3D structure of the scene, including the parallelepipeds themselves and other points, using the calibrated cameras and estimated poses.

A significant aspect is that the method can work with a small number of images and requires minimal user interaction, primarily for defining the parallelepiped vertices in the images.

Datasets

The paper evaluates its approach using:

  • Simulated images of a cube: Used to compare the proposed method with a vanishing point-based approach, demonstrating comparable or superior accuracy.
  • Real images of boxes with known dimensions: Used to estimate the accuracy of calibration and reconstruction under different scenarios.
  • Real images of a building: Used to demonstrate the system's ability to reconstruct complex scenes from a few images.

No new benchmarks were introduced; the focus was on evaluating the proposed method against existing techniques and on real-world scenarios.

Key Insights and Ideas

  • Parallelepipeds as powerful geometric primitives: The paper highlights that parallelepipeds, common in man-made environments, offer rich geometric constraints that can significantly simplify calibration and reconstruction.
  • Duality between camera and parallelepiped intrinsics: This is the central theoretical contribution. It allows treating cameras and parallelepipeds in a unified, dual framework, enabling the transfer of knowledge between them.
  • Linearization of calibration: By exploiting the duality and prior knowledge, the calibration problem can often be reduced to solving linear systems, which are computationally efficient and robust.
  • Reduced user interaction: The method requires only a few points to define a parallelepiped, making it practical for interactive modeling and for scenarios with limited data.
  • Robustness to noise: The use of geometric constraints and linear solutions is shown to be comparable to or better than non-linear optimization methods in certain situations, especially near singular configurations.

Key Results

  • The proposed method achieves comparable or better accuracy than vanishing point-based methods for camera calibration, particularly in near-singular situations (Figure 1).
  • Experimental results on known scenes show low average relative errors for estimated camera aspect ratios and parallelepiped length ratios across various calibration scenarios (Table 1).
  • The system demonstrates the ability to reconstruct 3D scenes from a small number of images, including non-overlapping ones, and to model complex structures like buildings (Figures 4 and 5).
  • The approach is shown to be effective even when only a few points are used to define the parallelepipeds.

Limitations and Future Work

  • User interaction: While reduced, user interaction is still required to define the parallelepiped primitives in the images.
  • Singularities: The paper acknowledges that singular cases can exist where calibration might not be achievable, and refers to prior work for details.
  • Future Work: The authors suggest several directions:
  • Automatization: Developing methods to automatically detect and define primitives in images, further reducing user interaction.
  • Refinement: Improving the accuracy of the initial reconstruction through non-linear optimization.
  • Surface Generation: Automatically generating surfaces from the reconstructed points to create more complete 3D models.

Summary updated

BibTeX
@inproceedings{wilczkowiak2002d,
  author = {Marta Wilczkowiak and Edmond Boyer and Peter F. Sturm},
  title = {{3D Modelling Using Geometric Constraints: A Parallelepiped Based Approach}},
  booktitle = {European Conference on Computer Vision},
  year = {2002},
  doi = {10.1007/3-540-47979-1\_15},
}
Figure