Constraints for interpretation of line drawings under perspective projection

Fatih Ulupinar, Ramakant NevatiaCvgip Image Underst 1991 · DOI

TL;DR

This paper extends traditional line drawing interpretation techniques from orthographic to perspective projection, introducing new constraints and a novel "convergent symmetry" concept. These advancements enable more robust and unique 3D shape recovery from single images, even for complex curved surfaces.

What Problem Does This Paper Address?

Inferring the 3D shape of an object from its 2D line drawing is a fundamental problem in computer vision. This task is inherently ambiguous because different 3D shapes can produce the same 2D contours under varying viewing conditions. While significant progress has been made using orthographic projection (where parallel lines in 3D remain parallel in the image), this is often an oversimplification. Perspective projection, which more accurately models how cameras see the world (objects further away appear smaller, parallel lines converge), introduces greater complexity. Traditional methods struggle with the added degrees of freedom and the loss of simple geometric properties like parallel lines remaining parallel. This paper aims to bridge this gap by developing and extending constraints specifically for perspective projection, offering a more general and powerful framework for 3D shape recovery.

Methods

The authors extend existing constraints like the "shared boundary constraint" and "orthogonality constraint" to the perspective projection setting. A key innovation is the introduction of "convergent symmetry." Unlike "skew symmetry" observed under orthographic projection, convergent symmetry describes a phenomenon where lines of symmetry in the 3D object, when projected perspectively, converge to a single point on the image plane. This convergence provides significantly stronger constraints, often leading to unique surface orientation recovery.

The paper also introduces a new representation for plane normals using a unit vector (p, q, r) with the constraint p² + q² + r² = 1, which can represent any plane orientation, unlike the gradient space representation used in orthographic projection. They derive new mathematical formulations for these constraints, including:

  • Shared Boundary Constraint: Relates the orientations of two intersecting planes using the image of their line of intersection.
  • Parallel Lines Constraint: Exploits the knowledge that two lines are parallel in 3D, allowing their 3D orientation to be determined from their image projections.
  • Orthogonality Constraint: Uses the knowledge that two lines are orthogonal in 3D to constrain plane orientations.
  • Convergent Symmetry: A novel symmetry concept where lines of symmetry converge to a point in the image. This is shown to provide unique surface normal recovery for planar objects.

The paper also discusses extensions to curved surfaces, particularly focusing on "parallel symmetry" and its application to conic surfaces.

Datasets

Not discussed in the paper. The paper focuses on deriving theoretical constraints and illustrating them with examples, rather than empirical evaluation on large datasets.

Key Insights and Ideas

  • Perspective Projection Offers More Information: Counterintuitively, the added complexity of perspective projection, when properly handled, can lead to more constrained and even unique interpretations compared to orthographic projection.
  • Convergent Symmetry is Powerful: The newly defined "convergent symmetry" is a significant advancement. Its property of lines of symmetry converging to a single point in the image provides much stronger cues for 3D shape recovery than orthographic "skew symmetry."
  • New Symmetry for Perspective: The paper effectively captures a visual phenomenon unique to perspective projection, which aligns better with human perception than the orthographic equivalent.
  • Generalizing Constraints: Successfully adapting and extending established orthographic constraints to the perspective domain is a crucial step for a more general 3D vision system.
  • Beyond Polyhedra: The work lays groundwork for analyzing curved surfaces, demonstrating how symmetry concepts can be extended to these more complex shapes.

Key Results

The paper provides a set of mathematical constraints for interpreting line drawings under perspective projection. While not presenting quantitative results from large-scale experiments, it demonstrates:

  • The derivation of generalized shared boundary, parallelism, and orthogonality constraints for perspective projection.
  • The definition and mathematical formulation of "convergent symmetry."
  • A constructive proof showing that convergent symmetry can uniquely determine the orientation of planar surfaces, unlike orthographic projection.
  • An example illustrating the application of these constraints to a cube, showing consistency between different constraint types.
  • An analysis of parallel symmetry for curved surfaces, particularly conic surfaces, and how it can be used for surface reconstruction.

Limitations and Future Work

  • Computational Complexity: The paper acknowledges that perspective constraints are more complex than orthographic ones, requiring more variables and potentially more intricate equations.
  • Regularity Assumptions: Many of the derived constraints (e.g., convergent symmetry, parallelism, orthogonality) rely on assumptions about the regularity of the 3D object. The paper notes that these regularities might be less apparent in perspective projection.
  • Handling "Errors": The shared boundary constraint is presented as always applicable (ignoring drawing errors), but other constraints require careful application when such regularities are not perfectly observed.
  • Human Preferences: The authors conjecture that humans might prefer interpretations corresponding to "right cylindrical surfaces" for curved objects, but this was not experimentally verified.
  • General Curved Surfaces: While extensions to conic surfaces are discussed, a comprehensive analysis for all types of curved surfaces is left for future work.
  • "Ad Hoc" Choices: For cases where constraints don't yield a unique solution, the paper mentions that "ad hoc choices" might be made, suggesting a need for more principled ways to resolve ambiguities.
  • Optimal Solutions: The authors have not investigated methods for finding "optimal" solutions when multiple constraints conflict or when evidence is overwhelming, suggesting this as a future research direction.

Summary updated

BibTeX
@article{ulupinar1991constraints,
  author = {Fatih Ulupinar and Ramakant Nevatia},
  title = {{Constraints for interpretation of line drawings under perspective projection}},
  journal = {Cvgip Image Underst},
  year = {1991},
  doi = {10.1016/1049-9660(91)90007-c},
}
Figure