A Quasi-Dense Approach to Surface Reconstruction from Uncalibrated Iages

Maxime Lhuillier, Long QuanHAL 2005

TL;DR

This paper introduces a "quasi-dense" approach to 3D surface reconstruction from uncalibrated images, significantly improving upon traditional sparse methods. By generating a denser set of 3D points and integrating both 3D point data and 2D image information, the method achieves more robust, accurate, and complete 3D models, even with fewer input images.

What Problem Does This Paper Address?

Traditional 3D reconstruction from images often relies on either "sparse" point features or "dense" stereo matching. Sparse methods, while good for camera pose estimation, produce too few points for detailed surface reconstruction. Dense methods, especially for uncalibrated images, can be computationally expensive, struggle with large baseline (separated) images, and often produce smoothed, incomplete surfaces. This paper addresses the gap of obtaining a high density of reliable 3D points from uncalibrated images, which is crucial for creating detailed surface models for applications like modeling and visualization.

Methods

The core innovation is the "quasi-dense" approach, which bridges the gap between sparse and dense methods.

  • Quasi-Dense Correspondence: Instead of relying solely on sparse interest points, the method first computes "quasi-dense pixel correspondences" using a match propagation technique. These are then refined by re-sampling using local homographies to achieve sub-pixel accuracy and a more uniform distribution, resulting in "quasi-dense point correspondences."
  • Geometry Estimation: A robust two-view geometry estimation is performed using these quasi-dense correspondences, followed by a fast, gauge-free uncertainty estimation to assess reconstruction accuracy.
  • Surface Reconstruction: A novel variational approach is proposed that integrates both the quasi-dense 3D point data and 2D image information (like photo-consistency and silhouettes). This is formulated as minimizing a functional using level-set methods, which naturally handles topological changes in the surface. A key component is a "bounded regularization method" that balances detail preservation with noise reduction.

Datasets

The paper evaluates its methods on several real-world image sequences, including:

  • Corridor sequence
  • Lady 1 sequence
  • Lady 2 sequence
  • Garden-cage sequence
  • Bust sequence
  • Bottle sequence
  • Apple sequence

The paper does not introduce new datasets but uses these diverse sequences to demonstrate the robustness and applicability of their approach across different scenarios.

Key Insights and Ideas

  • Bridging the Sparse-Dense Gap: The "quasi-dense" concept is a clever way to get the benefits of dense point clouds (for surface reconstruction) without the full computational cost or limitations of traditional dense stereo, especially in uncalibrated scenarios.
  • Uncalibrated Reconstruction Advantage: The quasi-dense approach is shown to be more effective than standard sparse methods for images with larger geometric distortions, requiring fewer images overall.
  • Unified Functional for Reconstruction: Integrating 3D point data and 2D image cues (like photo-consistency) within a single variational framework leads to more robust surface reconstruction, compensating for sparse 3D data.
  • Bounded Regularization for Detail: The proposed bounded regularization method is crucial for achieving a good balance between smoothing out noise and preserving fine geometric details, a common challenge in surface reconstruction.

Key Results

  • Robustness: The quasi-dense approach demonstrates significantly higher success rates in reconstruction compared to standard sparse methods, especially for challenging sequences with large image separations. For instance, a sequence that failed with sparse methods succeeded with the quasi-dense approach.
  • Accuracy: Uncertainty measures (confidence ellipsoids) for camera positions and 3D points are shown to be substantially smaller (3-6 times smaller for focal length and camera positions) for the quasi-dense method compared to sparse methods.
  • Efficiency: While the quasi-dense geometry estimation is more computationally intensive than sparse methods, it requires fewer images for a complete reconstruction. The surface reconstruction methods are also efficient, with some taking only a few minutes on a P4 2.4GHz processor.
  • Completeness: The method produces high-density 3D points, enabling the reconstruction of complete surfaces, which is a significant improvement over the partial surfaces often obtained with sparse methods.

Limitations and Future Work

  • Surface Assumptions: The surface reconstruction module is limited to smooth and closed objects. It is not designed for outdoor scenes or polyhedric objects.
  • Future Work: The authors suggest that the surface reconstruction module's limitations are due to the choice of the surface evolution approach. Future work could explore methods that can handle more complex object types.

Summary updated

Abstract

This paper proposes a quasi-dense approach to 3D surface model acquisition from uncalibrated images. Firstly, correspondence information and geometry are computed based on new quasi-dense point features that are re-sampled sub-pixel points from a disparity map. The quasi-dense approach gives more robust and accurate geometry estimations than the standard sparseapproach. The robustness is measured as the success rate of full automatic geometry estimation with all involved parameters fixed. The accuracy is measured by a fast gauge-free uncertainty estimation algorithm. The quasi-dense approach also works for more largely separated images than the sparse approach, therefore requires fewer images for modeling. More importantly, the quasi-dense approach delivers a high density of reconstructed 3D points on which a surface representation can be reconstructed. This fills the gap of insufficiency of the sparse approach for surface reconstruction, essential for modeling and visualization applications. Secondly, surface reconstruction methods from the given quasi-dense geometry are also developed. The algorithm optimizes new unified functionals integrating both 3D quasi-dense points and 2D image information including silhouettes. Combining both 3D data and 2D images is more robust than the existing methods using only 2D information or only 3D data. An efficient bounded regularization method is proposed to implement the surface evolution by level-set methods. Its properties are discussed and proved for some cases. As a whole, a complete automatic and practical system of 3D modeling from raw images captured by hand-held cameras to surface representation is proposed. Extensive experiments demonstrate the superior performance of the quasi-dense approach with respect to the standard sparse approach in robustness, accuracy, and applicability.
BibTeX
@inproceedings{lhuillier2005quasi,
  author = {Maxime Lhuillier and Long Quan},
  title = {{A Quasi-Dense Approach to Surface Reconstruction from Uncalibrated Iages}},
  booktitle = {HAL},
  year = {2005},
}

Loading related content...

Figure