Abstract
Reconstructing mirror surfaces is a significant challenge in computer vision due to their viewpoint-dependent appearance. This paper proposes a method to reconstruct 3D mirror surfaces using a single stationary, uncalibrated camera and a moving reference plane. By observing reflections of the plane in at least three unknown poses, the method recovers camera intrinsics, plane poses, and the mirror's 3D shape. The approach utilizes reflection correspondences and a cross-ratio based optimization to achieve high precision without tedious prior calibration.
Introduction
While 3D reconstruction of matte objects has reached high levels of maturity, specular surfaces like mirrors remain problematic. Standard techniques like Structure-from-Motion rely on the "brightness constancy constraint," which assumes a point's appearance is invariant to the viewing angle. Mirrors violate this, as their appearance is a reflection of the surrounding environment.
To address this, "Shape from Specular Reflection" techniques typically use a screen to display patterns reflected off the mirror. However, most existing methods require a fully calibrated setup, including precise knowledge of camera intrinsics and the exact 3D motion of the reference screen. This paper introduces a novel pipeline that removes these requirements, enabling mirror reconstruction in uncalibrated, "in-the-wild" settings.
Methodology
The proposed reconstruction pipeline leverages the geometric properties of light paths and projective invariants.
Reflection Geometry and the "Power of Three"
The core idea is to establish reflection correspondences between 3D points on an LCD screen and 2D pixels in the camera. By moving the screen to at least three unknown positions, the system gains enough geometric constraints to solve for the relative 3D motion of the screen and the camera parameters simultaneously.
As shown in the geometric model, light rays from points $X_0, X_1, X_2$ on planes $P_0, P_1, P_2$ reflect off the mirror surface $S$ at point $M$ and are captured at image point $m$ by camera $C$. The authors represent these light paths as 3D lines using Plücker coordinates, deriving a "line projection matrix" that can be converted into a standard camera projection matrix.
Cross-Ratio Optimization
To ensure robustness against noise and lens distortion, the method employs the cross-ratio—a fundamental invariant in projective geometry. The camera projection matrix is optimized by minimizing reprojection errors based on this cross-ratio constraint. This approach is significantly more stable than traditional point-to-line distance metrics, which often suffer from "sliding" errors in 3D space.
Experimental Evaluation
The method was validated through both synthetic simulations and real-world experiments.
Setup and Data
For real-world testing, the researchers used a 19-inch LCD monitor as the reference plane and a Canon EOS 40D digital camera. Objects such as a metallic sauce boat and chrome spheres were reconstructed.
Synthetic tests utilized standard models like the "Stanford Bunny" and an "engine hood" to evaluate performance against ground truth under varying noise levels.
Key Results
- Precision: Synthetic tests showed camera intrinsic errors (e.g., focal length) below 1%, with rotation errors as low as 0.08 degrees.
- Accuracy: Real-world reconstructions of objects (20-30cm in size) achieved a Root Mean Square (RMS) error consistently below 3mm.
- Robustness: The cross-ratio optimization effectively handled lens distortion, producing accurate 3D point clouds even without explicit initial radial distortion modeling.
Conclusion and Future Work
This research demonstrates that mirror surfaces can be reconstructed with high accuracy using uncalibrated equipment. By treating the problem as a light path triangulation, the mirror's viewpoint dependency becomes a geometric advantage.
However, limitations remain. The mathematical model can fail for "degenerate" shapes like perfect planes or specific elliptical curves. Additionally, the current method assumes single reflections, making it unsuitable for complex concave surfaces where inter-reflections occur. Future work may address these complexities and extend the method to non-perfectly specular (glossy) materials.
Summary updated