TL;DR
This paper introduces VortSDF, a novel method for 3D shape reconstruction that leverages Centroidal Voronoi Tessellation (CVT) to create an adaptive, non-uniform discretization of space. By optimizing signed distance fields (SDFs) on this adaptive grid, VortSDF achieves unprecedented reconstruction quality and detail, particularly in complex scenes, while maintaining competitive computational efficiency.
What Problem Does This Paper Address?
Reconstructing detailed 3D shapes from multiple images is a fundamental challenge in computer vision. While volumetric representations like voxel grids have been successful, they suffer from a precision-vs-complexity trade-off. Standard uniform grids are inefficient, dedicating resources to empty space, and struggle with fine details. Adaptive methods like octrees or hash maps can be complex to manage during optimization. This paper addresses the need for a discretization strategy that is both adaptive to shape geometry and computationally efficient, allowing for high-fidelity reconstruction without excessive resource consumption.
Methods
The core innovation of VortSDF lies in its use of Centroidal Voronoi Tessellation (CVT) to discretize the 3D space. Instead of a uniform grid, CVT creates a non-uniform partitioning of space where cells are optimally shaped and positioned around their centroids. This allows the method to focus computational resources on areas near the object's surface.
The framework jointly optimizes:
- An SDF field: This field represents the shape's geometry.
- A hierarchical CVT: This adaptive grid structure is optimized alongside the SDF.
- Two shallow color networks: These networks predict view-dependent color based on interpolated SDF and feature values.
A key aspect is the SDF-aware implicit CVT optimization. Instead of traditional, computationally expensive CVT methods, they propose an approximate loss function that encourages CVT sites to be equidistant from neighboring bisectors. This significantly speeds up CVT optimization.
The method employs a coarse-to-fine strategy. It starts with a coarse CVT and SDF, optimizes them, and then iteratively up-samples the CVT non-uniformly based on the SDF field, adding more detail around the surface. This process is guided by a differentiable rendering pipeline that efficiently samples points along rays within the tetrahedral mesh.
Datasets
The paper evaluates VortSDF on two public datasets:
- BlendedMVS: Used for reconstructing objects, open scenes, and human shapes.
- 4D Human Outfit: A dataset containing high-resolution 3D meshes of human subjects with calibrated cameras.
Key Insights and Ideas
- Adaptive Discretization is Crucial: The paper strongly demonstrates that adapting the spatial discretization to the geometry of the scene, rather than using uniform grids, is key to achieving high-frequency details and reducing artifacts.
- CVT as an Optimal Discretization: CVT provides a principled way to create an adaptive grid that naturally conforms to the shape, leading to better partitioning of the observation space around surfaces.
- Efficient Approximate CVT Optimization: The proposed SDF-aware implicit CVT optimization significantly reduces the computational cost of adapting the grid, making it feasible for large-scale reconstructions.
- Coarse-to-Fine Refinement: Iteratively refining the CVT grid based on the evolving SDF field allows for a progressive increase in detail, starting from a coarse representation and ending with a highly detailed surface.
Key Results
- Unprecedented Reconstruction Quality: VortSDF consistently achieves higher accuracy and completeness compared to state-of-the-art methods like NeuS2 and Voxurf, especially on complex and open scenes.
- Competitive Speed: While achieving higher quality, VortSDF converges significantly faster than methods like NeuS (30-50 minutes vs. 8.5 hours).
- Reduced Detail Budget: The method can achieve comparable or better results with substantially fewer points (e.g., 500K points at level 4 vs. 16M voxels for Voxurf).
- Fewer Artifacts: Qualitative results show VortSDF produces cleaner meshes with fewer discretization artifacts compared to other methods.
Limitations and Future Work
- Tetrahedral Mesh Computation: A current bottleneck is the computation of the tetrahedral mesh at each up-sampling level, which is done using an off-the-shelf CPU software (Open3D). The authors suggest that a GPU implementation of Delaunay triangulation would significantly speed up the process.
- Future Work: The authors envision further improvements by leveraging more advanced GPU implementations for tetrahedral mesh construction to achieve even faster speedups.
Summary updated