Abstract
We propose a paradigm for shape and motion reconstruction based on dynamic energy constraints. Objects are modeled as deformable elastic bodies and constraints derived from image data are modeled as external forces applied to these bodies. The external constraint forces are designed to mold a deformable body into a configuration that satisfies the constraints, making the model consistent with the images. We present a particular shape model whose internal forces induce a preference for surface continuity and axial symmetry. We develop a constraint force for dynamic stereo images and present results for the recovery of shape and non-rigid motion from natural imagery.
Introduction
This seminal 1987 work by Terzopoulos, Witkin, and Kass introduces a powerful paradigm for 3D reconstruction. By treating objects as deformable elastic bodies "molded" by image-derived forces, the authors demonstrate how to recover complex shapes and non-rigid motions directly from stereo video sequences, bypassing traditional intermediate representations.
Problem Statement
In the late 1980s, 3D reconstruction was often a fragmented process. Researchers struggled to integrate "low-level" image data (like edges) with "high-level" knowledge about object behavior (like physical continuity or symmetry).
A significant challenge in stereo vision was the "occluding boundary" problem: the silhouette of a smooth object appears different from different camera angles because the "horizon" of the object shifts. Traditional stereo matching often failed in these regions. Furthermore, most systems struggled with non-rigid motion—objects that bend, stretch, or deform over time.
Methodology
The authors propose a physics-based approach where an object is modeled as a deformable elastic body.
The Model Architecture
The model consists of a 1D "spine" (a space curve) inside a 2D "tube" (an elastic sheet). These components are coupled by virtual springs. This structure allows the model to represent a wide variety of shapes while maintaining a coherent topology.
Energy Constraints
The "shape" of the model is determined by minimizing an energy functional:
- Internal Forces: These represent physical properties like stiffness and tension. Crucially, the model is "symmetry-seeking"—it prefers axial symmetry (like a cylinder) but is flexible enough to deviate if the data demands it.
- External Forces: Derived from images, these forces are based on intensity gradients (silhouettes). They are de-projected into 3D space to create a force field that "pulls" the model’s boundaries toward observed edges.
Numerical Solution
The model is treated as a massless material in a viscous medium. The resulting equations of motion are solved using an efficient Alternating Direction Implicit (ADI) method, allowing the model to "evolve" or "flow" into the correct shape over several iterations.
Datasets and Evaluation
The method was evaluated on natural imagery:
- Still Life: A stereo pair of a potato and a pear, used to demonstrate handling of partial occlusions.
- Finger Sequence: A temporal sequence of 40 stereo video fields showing a human finger moving and bending, captured using a beam-splitting stereo adaptor.
Key Results
The system successfully reconstructed the 3D shape and non-rigid motion of a human finger. The "symmetry-seeking" nature allowed the model to produce plausible 3D volumes even when only silhouettes were available, effectively "hallucinating" the back side of the object based on its front-side symmetry.
Efficiency and Robustness
Initial reconstruction from a "cold start" (a simple cylinder) took approximately 40 iterations. Once initialized, the model tracked the finger through subsequent video frames in roughly 20 iterations per frame. By applying separate forces to the left and right boundary curves, the system effectively addressed the occluding boundary problem.
Discussion and Future Work
While successful, the current implementation relies heavily on silhouettes. It does not yet exploit internal surface details like shading or texture, which limits the detection of rotations around the object's own axis. Future work involves integrating shading and texture constraints into the energy functional and automating initialization using scale-space techniques.
Summary updated