Reconstructing a Visible Surface

Andrew BlakeAAAI 1984

Abstract

Early computer vision techniques for surface reconstruction from sparse stereo data relied heavily on "thin plate" interpolation models. However, these models are fundamentally flawed because they are not viewpoint-invariant, leading to a phenomenon known as the "wobble" effect. This paper identifies the mathematical roots of this instability and proposes a new framework for surface reconstruction based on intrinsic geometric properties—specifically curvature and surface area—that remains stable as the camera moves.

Introduction

In the early 1980s, a primary objective in computer vision was the interpolation of sparse data points, such as those obtained from stereo matching, into smooth, continuous surfaces. The prevailing methodology, championed by researchers like Grimson and Terzopoulos, modeled surfaces as "thin physical plates" that bend to intersect known data points. While mathematically convenient, this approach suffers from two major limitations: physical inaccuracy due to "small-slope" assumptions and a critical lack of viewpoint invariance.

The Problem of Non-Invariance: The "Wobble" Effect

The most significant drawback of standard interpolation is that the resulting math is tied to the viewer's coordinate system (the $z$-axis). Consequently, the reconstructed shape of an object changes depending on the angle from which it is viewed. For a robot or autonomous system, this means a static object would appear to deform or "wobble" as the sensor moves, making reliable spatial reasoning impossible.

Demonstration of the 'wobble' effect: as the viewer direction changes, the interpolated curve between fixed points deforms.

As shown in the figure above, interpolating between fixed points using standard splines results in different curves as the viewer's orientation shifts. This demonstrates that the "best fit" surface in traditional models is a function of the camera's position rather than the object's actual geometry.

Methodology: From Extrinsic to Intrinsic Geometry

To resolve this, the paper critiques standard Biharmonic Interpolation, which seeks to minimize "quadratic variation" (the integral of the sum of second derivatives, $f_{xx}^2 + f_{yy}^2$). While this approach leads to the manageable biharmonic equation ($\Delta^2 f = 0$), it is inherently extrinsic.

Blake proposes shifting toward a Viewpoint-Invariant Energy functional based on the intrinsic properties of the surface itself. Instead of projecting onto a 2D grid, the proposed method minimizes the sum of the squares of the principal curvatures ($k_1^2 + k_2^2$) integrated over the actual surface area ($dS$).

Because this invariant energy is non-linear and non-convex, it is significantly more difficult to solve than linear models. Blake suggests an optimization strategy that incorporates a "stretching" term—analogous to a flexible rod under tension—and utilizes gradient descent and relaxation techniques within specific slope constraints to converge on a unique, stable solution.

Key Results and Insights

The paper provides a mathematical proof demonstrating that while quadratic variation is isotropic in 2D (invariant to rotation in the image plane), it fails to remain invariant when the depth ($z$-axis) changes. By contrast, the proposed energy functional—derived in 2D as $F = \int E(f_x, f_{xx}) ds$—depends only on the surface's own arc length and curvature.

Qualitatively, this method eliminates the "sagging" and "wobbling" characteristic of biharmonic interpolation. It moves away from the thin plate metaphor, which Blake argues is physically misleading because real plates possess stretching energy that penalizes certain geometric shapes. Instead, the focus is placed on a purely geometric ideal that respects 3D rotation.

Limitations and Future Directions

While the theoretical foundation is robust, the computational complexity of the invariant energy functional is a hurdle. Solving the resulting non-linear equations is far more intensive than the linear systems used in previous methods. Furthermore, while the 2D simulations successfully demonstrate the principle, extending these results to full 3D surfaces and addressing the complexities of occlusion—where moving the camera changes which data points are visible—remains a necessary area for future research.

Summary updated

BibTeX
@inproceedings{blake1984reconstructing,
  author = {Andrew Blake},
  title = {{Reconstructing a Visible Surface}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1984},
}
Figure