Energy Constraints on Deformable Models: Recovering Shape and Non-Rigid Motion

Demetri Terzopoulos, Andrew P. Witkin, Michael KassAAAI 1987 · OpenReview

Abstract

We propose a paradigm for shape and motion reconstruction based on dynamic energy constraints. Objects are modeled as deformable elastic bodies and constraints derived from image data are modeled as external forces applied to these bodies. The external constraint forces are designed to mold a deformable body into a configuration that satisfies the constraints, making the model consistent with the images. We present a particular shape model whose internal forces induce a preference for surface continuity and axial symmetry. We develop a constraint force for dynamic stereo images and present results for the recovery of shape and non-rigid motion from natural imagery.

Introduction

This seminal 1987 work by Terzopoulos, Witkin, and Kass introduces a powerful paradigm for 3D reconstruction. By treating objects as deformable elastic bodies "molded" by image-derived forces, the authors demonstrate how to recover complex shapes and non-rigid motions directly from stereo video sequences, bypassing traditional intermediate representations.

Problem Statement

In the late 1980s, 3D reconstruction was often a fragmented process. Researchers struggled to integrate "low-level" image data (like edges) with "high-level" knowledge about object behavior (like physical continuity or symmetry).

A significant challenge in stereo vision was the "occluding boundary" problem: the silhouette of a smooth object appears different from different camera angles because the "horizon" of the object shifts. Traditional stereo matching often failed in these regions. Furthermore, most systems struggled with non-rigid motion—objects that bend, stretch, or deform over time.

Methodology

The authors propose a physics-based approach where an object is modeled as a deformable elastic body.

The Model Architecture

The model consists of a 1D "spine" (a space curve) inside a 2D "tube" (an elastic sheet). These components are coupled by virtual springs. This structure allows the model to represent a wide variety of shapes while maintaining a coherent topology.

The model architecture consists of a 1D spine within a 2D elastic tube, mapped from a parametric domain.

Energy Constraints

The "shape" of the model is determined by minimizing an energy functional:

  • Internal Forces: These represent physical properties like stiffness and tension. Crucially, the model is "symmetry-seeking"—it prefers axial symmetry (like a cylinder) but is flexible enough to deviate if the data demands it.
  • External Forces: Derived from images, these forces are based on intensity gradients (silhouettes). They are de-projected into 3D space to create a force field that "pulls" the model’s boundaries toward observed edges.

Numerical Solution

The model is treated as a massless material in a viscous medium. The resulting equations of motion are solved using an efficient Alternating Direction Implicit (ADI) method, allowing the model to "evolve" or "flow" into the correct shape over several iterations.

Datasets and Evaluation

The method was evaluated on natural imagery:

  • Still Life: A stereo pair of a potato and a pear, used to demonstrate handling of partial occlusions.
  • Finger Sequence: A temporal sequence of 40 stereo video fields showing a human finger moving and bending, captured using a beam-splitting stereo adaptor.

Key Results

The system successfully reconstructed the 3D shape and non-rigid motion of a human finger. The "symmetry-seeking" nature allowed the model to produce plausible 3D volumes even when only silhouettes were available, effectively "hallucinating" the back side of the object based on its front-side symmetry.

Reconstruction and tracking of a non-rigid finger sequence over multiple frames.

Efficiency and Robustness

Initial reconstruction from a "cold start" (a simple cylinder) took approximately 40 iterations. Once initialized, the model tracked the finger through subsequent video frames in roughly 20 iterations per frame. By applying separate forces to the left and right boundary curves, the system effectively addressed the occluding boundary problem.

Discussion and Future Work

While successful, the current implementation relies heavily on silhouettes. It does not yet exploit internal surface details like shading or texture, which limits the detection of rotations around the object's own axis. Future work involves integrating shading and texture constraints into the energy functional and automating initialization using scale-space techniques.

Summary updated

Abstract

We propose a paradigm for shape and motion reconstruction based on dynamic energy constraints. Objects are modeled as deformable elastic bodies and constraints derived from image data are modeled as eztemal forces applied to these bodies. The eztemal constraint forces are designed to mold a deformable body into a configuration tent satisfies the constraints, making the model consistent with the images. We present a particular shape model whose internal forces induce a preference for surface continuity and axial symmetry. We develop a constraint force for dynamic stereo images and present results for the recovery of shape and non-rigid motion from natural imagery.
BibTeX
@inproceedings{terzopoulos1987energy,
  author = {Demetri Terzopoulos and Andrew P. Witkin and Michael Kass},
  title = {{Energy Constraints on Deformable Models: Recovering Shape and Non-Rigid Motion}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1987},
  url = {https://openreview.net/forum?id=HJVs8y-u-S},
}
Figure