When algorithms attempt to reconstruct the 3D world from 2D images, they face a fundamental challenge: the problem is severely under-constrained. To bridge the gap between flat pixels and three-dimensional space, computer vision systems rely heavily on geometric priors.
For decades, the most dominant and successful prior has been the piecewise planar assumption. This hypothesis posits that the world is largely composed of flat surfaces meeting at discrete boundaries. Mathematically, a plane is incredibly cheap to represent—requiring only four coefficients ($ax + by + cz + d = 0$)—which drastically reduces the degrees of freedom an optimization algorithm must navigate.
In man-made environments, this assumption works remarkably well. For instance, Bódis-Szomorú et al. demonstrated that treating urban scenes as piecewise planar allows for ultra-fast 3D reconstructions from sparse data, turning hours of computation into seconds. Similarly, Van Gool leveraged a piecewise planarity prior in monocular depth estimation to enforce sharp, clean boundaries and consistent geometry around walls and floors.
However, the real world is rarely perfectly flat. When a system strictly optimized for planes encounters the organic complexity of nature or the sweeping curves of modern architecture, the piecewise planar assumption transforms from a helpful constraint into a destructive limitation.
The Failure Modes of Flatness
What happens when a planar model meets a non-planar world? The system is forced to approximate curves using a patchwork of flat facets. This geometric under-parameterization leads to several distinct failure modes across different vision tasks.
1. Structural Distortion and Artifacts
When forced to reconstruct unstructured or organic geometry—such as trees, foliage, or statues—planar models introduce severe structural distortions. Freifeld et al. observed that locally planar primitives fail to capture the intricate, high-frequency details of natural environments. Instead of smooth transitions, the resulting models suffer from what Xie et al. describe as "blocky artifacts," where organic shapes are unnaturally squared off and flattened.
True Curved Surface: ~~~~~~~~~~~~~~~~~~~~~~~~~
Planar Approximation: |___|___|___|___|___|___|
(Results in blocky artifacts)
2. Downstream Tracking and Motion Errors
The errors introduced by ill-fitting planar assumptions degrade tracking and navigation. In egomotion estimation, Jepson et al. showed that assuming environmental surfaces are locally planar introduces significant errors when navigating highly curved layouts.
A similar breakdown occurs in video stabilization. Zhou et al. developed stabilization techniques based on large planar surfaces like walls and floors, but noted that the model fails in environments with significant curvature, such as rounded tunnels. Because the system cannot accurately parameterize the tunnel's curve, these regions are left susceptible to severe warping and wobble artifacts.
3. Material and Reflection Misalignment
The planar bias even distorts how systems interpret light. In neural rendering and reflection decomposition, Guo et al. highlighted that existing models often assume reflective surfaces are flat. When these algorithms encounter highly curved reflective objects—such as a polished metallic sphere—the planar assumption breaks down, causing the system to miscalculate depth and generate "ghostly" visual artifacts.
Moving to Higher-Order Geometries
To overcome these limitations, researchers are developing frameworks that explicitly model non-planar regularities. The goal is to retain the computational efficiency of low-parameter primitives while expanding the vocabulary of the optimization models to include curves.
A prominent example of this shift is the integration of quadric surfaces (such as spheres, cylinders, and cones) directly into spatial mapping pipelines. Davison et al. introduced a minimal representation for quadrics within Simultaneous Localization and Mapping (SLAM) systems. By allowing the system to recognize a column as a cylinder or a light fixture as a sphere, the robot can map common indoor environments with far less data than dense point clouds require, without suffering from the distortions caused by forcing those shapes into flat planes.
Primitive Type Parameters Required Handles Curves?
----------------------------------------------------------
Plane 4 No
Quadric Surface 9 (e.g., Cylinder) Yes
Similarly, in single-image reconstruction, Yang et al. emphasized the necessity of moving past planar-only segmentation to properly parameterize pillars and rounded facades. By allowing the underlying loss functions to understand continuous curvature, the models bridge the representational gap between rigid, rectilinear abstractions and the fluid geometry of the real world.