A volumetric fusion technique for surface reconstruction from silhouettes and range data

Yücel Yemez, C. J. WetheriltCVIU 2007 · DOI

TL;DR

This paper introduces a novel hybrid approach to 3D surface reconstruction that combines the strengths of two existing techniques: shape-from-silhouette (SFS) and shape-from-optical-triangulation (laser scanning). By volumetrically fusing data from both methods, they achieve more complete and accurate 3D models, particularly for objects with complex cavities and occlusions that challenge individual methods.

What Problem Does This Paper Address?

Creating accurate 3D models of objects is crucial for many applications, from virtual reality to cultural heritage preservation. Two common approaches exist: active methods like optical triangulation (using lasers) and passive methods like shape-from-silhouette.

Optical triangulation provides precise point clouds but often suffers from holes and missing data due to occlusions and the reflective properties of surfaces. Shape-from-silhouette, on the other hand, generates robust, hole-free "visual hulls" but can't capture internal cavities or fine details. This leaves a gap: how to get the best of both worlds – the completeness of SFS and the detail of optical triangulation – to create truly accurate and watertight 3D models.

Methods

The core of this work is a volumetric fusion technique that leverages an octree representation. Here's a breakdown:

  1. Octree Representation from Silhouettes: First, the system generates a volumetric representation of the object's visual hull using shape-from-silhouette. This is stored in an octree, where each node (voxel) can be classified as inside (IN), outside (OUT), or intersecting the surface (ON).
  2. Volume Carving with Range Data: The precise but incomplete range data from optical triangulation is then used to "carve" this octree. The algorithm processes scan lines from the laser data. If a scan line passes through a voxel that should be empty (based on the range data), that voxel is marked as OUT, effectively filling in cavities or correcting inaccuracies in the initial silhouette model. Special handling is included for outlier range data and to prevent erroneous carving.
  3. Hole Filling: After carving, some holes might still exist, especially around deep cavities. A hole-filling algorithm is applied, converting certain IN/OUT neighboring voxels to a new ON type to patch these gaps.
  4. Surface Triangulation (Marching Cubes): Finally, the modified octree is triangulated into a surface mesh using the marching cubes algorithm. To ensure a smooth and manifold surface, an "isosurface merging" technique is employed. This involves calculating isovalues at grid points by averaging local information from both silhouette and range data, effectively blending the two sources of geometric information.

What's novel here is the systematic way they combine these techniques in a volumetric framework, specifically addressing the challenges of merging isosurfaces from different data types and handling cavities.

Datasets

The paper evaluates their method on four real objects:

  • Greek1 (stone statuette)
  • Greek2 (stone statuette)
  • Elephant (wood statuette)
  • Hand (plastic statuette)

The authors do not introduce new datasets but rather use their custom acquisition setup to capture the necessary silhouette and range data for these objects.

Key Insights and Ideas

  • Complementary Strengths: Silhouettes provide a robust, hole-free outer shell, while range data fills in details and cavities. Combining them is more than the sum of their parts.
  • Volumetric Fusion is Key: Treating the problem volumetrically (using an octree) avoids many topological issues that can arise from purely surface-based fusion methods.
  • Carving with Scan Lines: Using the scan lines from optical triangulation, not just the points, is crucial for accurately carving out occluded or concave regions.
  • Isosurface Merging for Smoothness: The method for merging isovalues from different data sources is critical for producing a clean, hole-free final mesh.

Key Results

  • The proposed fusion technique produces robust and satisfactory surface reconstructions.
  • It successfully generates cavity-sensitive and hole-free models, even for objects with severe occlusions and sharp hollows (like the Hand and Greek1 statuettes).
  • Quantitative results (Table 2) show a significant reduction in the average distance of range data to the reconstructed surface compared to silhouette-only reconstruction. For example, for the Greek2 object, the average distance improved from 1.046 to 0.061.
  • The method is demonstrated to be effective even when the optical triangulation scanner has limitations (e.g., single-stripe, rotational-only movement).

Limitations and Future Work

  • Hardware Dependency: The performance is somewhat limited by the hardware setup. A fixed, single-stripe rotational scanner can lead to irregular sampling and limits the octree resolution that can be used without artifacts.
  • Future Work: The authors suggest that using range scanners capable of full-range images (supporting translational movement) could significantly improve data distribution and allow for higher octree resolutions, leading to sharper fusion results and better detail preservation. They also note that the execution time, while reasonable, is not fully optimized.

Summary updated

BibTeX
@article{yemez2007volumetric,
  author = {Yücel Yemez and C. J. Wetherilt},
  title = {{A volumetric fusion technique for surface reconstruction from silhouettes and range data}},
  journal = {CVIU},
  year = {2007},
  doi = {10.1016/j.cviu.2006.07.008},
}
Figure