When a three-dimensional world is captured by a single camera, an entire dimension of physical reality is discarded. Mathematically, a 3D point $(X, Y, Z)$ projects onto a 2D image plane via perspective scaling, meaning an infinite number of points along a single ray originating from the camera lens map to the exact same pixel coordinate.
This loss of the depth dimension creates what researchers call the monocular depth perception gap or monocular depth ambiguity. Without native depth information, systems relying solely on standard RGB video feeds cannot definitively resolve how far away an object is based on its pixel position alone. As noted by Lyu et al., this lack of native depth perception fundamentally constrances spatial localization and environmental mapping compared to binocular or active configurations.
The Depth Axis Error in Visuomotor Control
For a robot interacting with its environment, this geometric ambiguity introduces severe operational challenges. Consider a robotic arm attempting to grasp an object. In a monocular setup, the robot can easily determine the target's horizontal and vertical position ($X$ and $Y$) within the image frame. However, its estimation of the distance ($Z$) along the camera's optical axis remains highly uncertain.
Rahrakhshan et al. demonstrate that relying on a single camera for visuomotor reaching tasks introduces significant localization errors specifically along the depth axis, causing the robot to overshoot or undershoot its target. This spatial ambiguity makes precise physical interaction incredibly difficult.
[Camera] ───► ───► ───► [Ray of Ambiguity] ───► ───► ───►
? (Is the object at 1m, 2m, or 3m?)
This limitation ripples across various manipulation domains:
- Object Grasping: Bauer et al. highlight that depth perception limitations hinder the development of effective marker-less grasping strategies, while Bousmalis et al. observe that omitting depth information limits the precision of spatial reasoning needed for industrial manipulation.
- Policy Learning: Even when scaling advanced reinforcement learning models, Zeng et al. point out that while RGB inputs may suffice for basic navigation, the lack of geometric depth information severely restricts an agent's capability to perform complex object manipulation.
- Micro-scale Manipulation: In specialized domains like micromanipulation, Sha et al. identify that monocular vision introduces intrinsic "blind areas" and single-point target control limitations.
The Cognitive Cost on Human Operators
The monocular depth gap does not just affect autonomous algorithms; it also severely impacts human operators in teleoperation and robotic surgery. When a surgeon or vehicle pilot controls a machine through a standard 2D monitor, they lose natural depth cues like stereopsis and motion parallax.
In minimally invasive surgery, standard monocular endoscopes eliminate natural depth cues. The authors of MULTI-CAMERA LAPAROSCOPIC IMAGING WITH TUNABLE FOCUSING CAPABILITY note that this single imaging source complicates tasks requiring precise distance estimation, such as suturing. To compensate, operators must rely on indirect, secondary cues. For instance, Kim et al. explain that in top-down microsurgical environments, surgeons must track the convergence of a tool with its own shadow to estimate distance to delicate tissues—a process that is highly cognitively demanding. Schütz et al. confirm that relying on traditional 2D microscopy forces a heavy reliance on these indirect cues, limiting precision in micrometer-scale procedures and increasing the operator's mental workload.
Similarly, Mayer et al. and Hein et al. emphasize that relying exclusively on monocular RGB input limits the precision of 3D localization for surgical instruments and hands, creating significant navigation challenges.
This issue extends directly to vehicle teleoperation. Luo et al. (2021) and Luo et al. (2022) demonstrate that remote operators using monoscopic feeds or head-mounted displays consistently struggle to judge distances and object velocities, leading to elevated collision rates. Stotko et al., Lee et al., and Komatsu emphasize that this "tunnel vision" and lack of depth information hinders high-precision tasks in complex or unfamiliar environments, as operators cannot accurately gauge the distance between the robot and surrounding obstacles.
High-Speed Navigation and Decision Making
In autonomous driving and mobile robot navigation, the depth gap translates directly into safety risks. A vehicle traveling at high speed needs to know exactly how far away an obstacle is to calculate the time-to-collision and plan a safe trajectory.
While end-to-end models can map monocular images directly to control commands, Wu et al. show that single-camera architectures still lack the depth resolution required for high-fidelity spatial reasoning compared to active sensors like LiDAR. This limitation can degrade object detection and route completion. Lübberstedt et al. emphasize that extracting accurate depth from monocular 2D video feeds remains a fundamental bottleneck, hindering an autonomous system's ability to make informed decisions based on physical proximity. When depth perception fails, as Wnuk et al. observe, robots require complex geometric calculations or supplemental sensing to achieve reliable long-range navigation.
Bridging the Gap
To overcome the mathematical limitations of a single 2D viewpoint, modern research explores several distinct pathways:
- Artificially Restoring Cues: Miura et al. combat the lack of depth in monocular laparoscopy by tracking tools in real-time and projecting "virtual shadows" onto organ surfaces, drastically reducing surgical errors by restoring a familiar spatial cue.
- Multi-View and Stereo Synthesis: Moving from monocular to binocular or multi-camera arrays provides the baseline distance necessary for triangulation. For example, the authors of An Agricultural Spraying Robot Based on the Machine Vision leverage binocular vision to achieve the precise 3D localization required for targeted manipulation in complex agricultural environments.
- Advanced 3D Reconstructions: Rather than forcing operators to interpret flat video feeds, frameworks like the one proposed by Boehringer et al. use techniques like Gaussian Splatting to build high-fidelity 3D environments from remote data, bypassing the monocular limitation entirely and allowing operators to interact intuitively within a spatially accurate virtual space.
Key Foundational Work
Integration of Force Feedback in an Open Robot Platform for Robotic Surgery (2004) — contributed 1 concept elements to this direction.
A Multi Agent Approach to Vision Based Robot Scavenging (2006) — contributed 1 concept elements to this direction.