Introduction
Stereoscopic 3D content is increasingly prevalent across various platforms, from cinema to VR. However, the optimal depth range varies significantly depending on the display size and viewer comfort. Disparity remapping is the essential process of adjusting these depth levels. The primary challenge lies in modifying depth without distorting the underlying 3D geometry or 2D shapes of objects.
Existing methods often treat images as simple 2D planes. When disparity is stretched or squashed, it can inadvertently "warp" objects—for instance, turning a flat wall into a curved one. This paper addresses these issues by balancing three conflicting goals: reaching target depth, preserving 2D shapes, and maintaining 3D structural integrity.
The 3D Structure-Preserving Constraint
The core innovation of this work is a new energy term ($E^G$) that enforces 3D consistency. The fundamental insight is that if corresponding regions in the left and right images are warped identically (consistently), the 3D structure of the object is preserved.
As shown in the conceptual diagram above, inconsistent warping leads to distorted 3D geometry. The authors measure "warping inconsistency" by comparing the edge vectors of warped grid meshes between the two views, mathematically penalizing any asymmetry to keep 3D structures "flat" or "straight" as intended.
Two-Stage Warping Methodology
The authors propose a two-stage approach formulated as a constrained optimization problem to handle the trade-off between stability and flexibility.
Stage I: Axis-Aligned Warping
The first stage employs axis-aligned warping, which constrains warped grids to remain rectangular. This approach is highly robust, preventing "foldovers" and ensuring a convex optimization problem. This stage focuses on bringing salient objects to their target depths while maintaining their 2D shapes and 3D structures perfectly.
Stage II: Vertex-Based Warping
While axis-aligned warping is stable, its rigidity can cause distortions in background regions. In the second stage, the algorithm switches to vertex-based warping specifically for these less important areas. Using the results from Stage I as an initial guess, background vertices are allowed to move more freely to smooth out distortions introduced by the rigid first stage.
Key Results and Performance
The method was evaluated using standard benchmarks (Flickr, Middlebury), movie content (The Monkey King 2, Monster Hunt), and synthetic frames (Elephants Dream).
Qualitative comparisons demonstrate that while previous state-of-the-art methods like Lang et al. often "bend" the 3D scene during remapping, this two-stage approach maintains straight lines and accurate depth. Quantitatively, the method achieved a 3D structure preservation score of 0.876, significantly outperforming VWR (0.654) and DWR (0.672).
Furthermore, a subjective study with 31 participants showed that over 71% of users preferred this method over existing techniques. The optimization is also efficient, taking approximately 0.3 seconds per frame, making it nearly 5x faster than some competing warping-based methods.
Conclusion
By combining the stability of axis-aligned warping with the flexibility of vertex-based refinement, this two-stage approach effectively solves the problem of geometric distortion in disparity remapping. The key lies in the mathematical enforcement of warping consistency between views, ensuring that the 3D world remains natural even after significant depth manipulation.
Summary updated