Implicit meshes : unifying implicit and explicit surface representations for 3D reconstruction and tracking

Slobodan IlićInfoscience 2005 · DOI

Abstract

This research proposes a novel surface representation, termed the "implicit mesh," designed to bridge the gap between traditional explicit surfaces (like triangulated meshes) and implicit surfaces (defined by potential fields). While explicit meshes are the industry standard for rendering and animation, they often present non-differentiable distance functions that complicate automated fitting. Conversely, implicit surfaces offer differentiable algebraic distances suitable for optimization but are difficult to deform and render intuitively. By wrapping a differentiable potential field around a standard triangulated mesh, this method combines the strengths of both representations. We further introduce Dirichlet Free Form Deformation (DFFD) as a generic parameterization technique to reduce problem dimensionality during optimization. The resulting framework enables robust 3D reconstruction and tracking from noisy stereo and silhouette data, even in cluttered environments with significant occlusions.

Introduction

In the fields of computer vision and graphics, representing 3D objects typically involves a choice between two paradigms: explicit and implicit representations. Explicit models, such as triangulated meshes, are highly efficient for rendering and manual editing. However, when attempting to fit these models to image data—a process known as "Shape from X"—the distance function between data points and the mesh surface is often non-differentiable, leading to optimization failures.

Implicit surfaces, defined by a function $F(x, y, z) = 0$, provide a mathematically smooth landscape that is ideal for gradient-based optimization. Yet, they lack the intuitive structure required for complex animations and fast rendering. This paper introduces the Implicit Mesh, a hybrid representation that uses the explicit mesh as a structural skeleton and an implicit "shell" for optimization, providing a unified tool for automated shape recovery and tracking.

The Implicit Mesh Framework

The core innovation of this work is the construction of a $C^1$ continuous potential field around a standard mesh. This is achieved through the use of Triangular Metaballs. Unlike traditional spherical metaballs, which can create a "bumpy" or irregular surface on low-resolution models, these triangular primitives create a thin, smooth volumetric layer that follows the facets of the mesh.

This creates a "tandem deformation" effect: as the underlying explicit mesh deforms, the implicit shell deforms with it. This allows researchers to use the implicit function's differentiable properties during the fitting phase (using algorithms like Levenberg-Marquardt) while retaining the mesh for final rendering and standard graphics pipelines.

Efficient Parameterization with DFFD

To make the reconstruction process computationally feasible, the dimensionality of the deformation must be managed. Adjusting every vertex in a high-resolution mesh individually is inefficient and prone to noise.

The author advocates for Dirichlet Free Form Deformation (DFFD). DFFD utilizes a sparse set of non-grid control points and Sibson coordinates (natural neighbors) to parameterize the object's shape. By moving only these control points, the entire mesh deforms smoothly. This approach drastically reduces the number of parameters the optimization algorithm must solve, making the fitting process both faster and more stable.

Robust Silhouette Detection via ODEs

One of the most challenging aspects of 3D reconstruction is accurately utilizing silhouettes (occluding contours). Traditional methods often rely on discrete searches or "snakes" that can be easily distracted by high-contrast background clutter.

By leveraging the implicit mesh, the author treats silhouette detection as solving an Ordinary Differential Equation (ODE). This allows the system to find perfectly smooth contours that are independent of the mesh's polygon count. Because the silhouette is tied to the 3D geometry of the model, the system can effectively ignore background noise and handle partial occlusions—such as a hand passing in front of a face—that would typically cause silhouette-based methods to fail.

Experimental Results

The framework was validated across several complex scenarios:

  • Complex Geometries: The method successfully reconstructed the intricate folds of a human ear from noisy data, achieving a significantly lower mean distance error than traditional mesh-fitting techniques.
  • Human Anatomy: The system was applied to the reconstruction of the human upper body (face, neck, and shoulders) from uncalibrated video sequences. It also demonstrated compatibility with existing statistical models, such as PCA-based face models.
  • Non-Rigid Tracking: The research showcased the tracking of deformable objects in monocular video, including the bending of a piece of paper and the shrugging of a person's shoulders. Even with cluttered backgrounds, the implicit mesh maintained a stable fit where traditional trackers often lost the target.

Conclusion and Future Directions

The implicit mesh represents a significant step forward in unifying the mathematical elegance of implicit functions with the practical utility of explicit meshes. By providing a differentiable bridge between the two, it enables more robust and automated 3D reconstruction from real-world, noisy visual data.

Future work may involve generalizing the triangular metaball concept to other polygonal types, such as quads or NURBS. Additionally, integrating physical constraints through Finite Element Methods (FEM) could further improve the realism of deformations, ensuring that tracked movements adhere to the physical properties of the materials being modeled, such as the elasticity of human skin.

Summary updated

Abstract

This thesis proposes novel ways both to represent the static surfaces, and to parameterize their deformations. This can be used both by automated algorithms for efficient 3–D shape reconstruction, and by graphics designers for editing and animation. Deformable 3–D models can be represented either as traditional explicit surfaces, such as triangulated meshes, or as implicit surfaces. Explicit surfaces are widely accepted because they are simple to deform and render, however fitting them involves minimizing a non-differentiable distance function. By contrast, implicit surfaces allow fitting by minimizing a differentiable algebraic distance, but they are harder to meaningfully deform and render. Here we propose a method that combines the strength of both representations to avoid their drawbacks, and in this way build robust surface representation, called implicit mesh, suitable for automated shape recovery from video sequences. This surface representation lets us automatically detect and exploit silhouette constraints in uncontrolled environments that may involve occlusions and changing or cluttered backgrounds, which limit the applicability of most silhouette based methods. We advocate the use of Dirichlet Free Form Deformation (DFFD) as generic surface deformation technique that can be used to parameterize objects of arbitrary geometry defined as explicit meshes. It is based on the small set of control points and the generalized interpolant. Control points become model parameters and their change causes model's shape modification. Using such parameterization the problem dimensionality can be dramatically reduced, which is desirable property for most optimization algorithms, thus makes DFFD good tool for automated fitting. Combining DFFD as a generic parameterization method for explicit surfaces and implicit meshes as a generic surface representation we obtained a powerfull tool for automated shape recovery from images. However, we also argue that any other avaliable surface parameterization can be used. We demonstrate the applicability of our technique to 3–D reconstruction of the human upper-body including – face, neck and shoulders, and the human ear, from noisy stereo and silhouette data. We also reconstruct the shape of a high resolution human faces parametrized in terms of a Principal Component Analysis model from interest points and automatically detected silhouettes. Tracking of deformable objects using implicit meshes from silhouettes and interest points in monocular sequences is shown in following two examples: Modeling the deformations of a piece of paper represented by an ordinary triangulated mesh; tracking a person's shoulders whose deformations are expressed in terms of Dirichlet Free Form Deformations.
BibTeX
@inproceedings{ilic2005implicit,
  author = {Slobodan Ilić},
  title = {{Implicit meshes : unifying implicit and explicit surface representations for 3D reconstruction and tracking}},
  booktitle = {Infoscience},
  year = {2005},
  doi = {10.5075/epfl-thesis-3243},
}
Figure