Bridging the Exteroceptive Sensing Gap in Robotic Locomotion

385 concepts266 papers2 proposals

Modern robotics has achieved remarkable success in controlled environments by mastering proprioception—the internal sense of a robot’s own body. By monitoring joint angles, motor torques, and inertial measurements, robots can learn to balance, walk, and even recover from significant pushes. However, a fundamental "sensing gap" remains: these systems are often effectively blind. Without exteroception—the ability to perceive the external environment through vision, LiDAR, or depth sensors—robots are restricted to reactive behaviors, unable to anticipate hazards or navigate complex, non-random terrain.

The Proprioceptive Bedrock

The current state of the art in legged locomotion relies heavily on high-frequency proprioceptive feedback. As Yuxiang Yang et al. demonstrate, a quadruped can learn to walk in just minutes of real-world data using only internal sensors. This approach is robust because proprioceptive data is reliable, low-latency, and transfers easily from simulation to reality.

In fact, some systems push proprioception to its limits to avoid the complexity of vision entirely. Haozhi Qi et al. developed the HORA system, which allows a multi-fingered hand to rotate diverse objects using only joint encoders. By "feeling" the object's properties and adapting gaits in real-time, the robot compensates for the lack of visual or tactile sensors. Similarly, In Hur et al. showed that a humanoid could balance on a tilting platform without force sensors by estimating the Zero Moment Point (ZMP) through motor current alone.

The Reactive Constraint

While proprioception provides stability, it imposes a "reactive constraint." A blind robot cannot see a hole, a step, or an obstacle; it can only react once it physically makes contact with them. Runhan Huang et al. argue that this "blind locomotion" approach necessitates a strategy where the system must experience a disturbance before adapting.

This lack of anticipation leads to several failure modes:

  • Terrain Discontinuities: As Ruben Grandia et al. point out, systems relying on internal kinematic limits cannot identify gaps or pits until a foot falls into one.
  • Gait Inefficiency: Wenjuan Ouyang notes that without exteroceptive data, robots cannot proactively adjust their gait for upcoming terrain shifts, forcing less efficient, purely reactive transitions.
  • Obstacle Collisions: In manipulation, Jonas Stolle et al. show that relying only on joint sensors prevents a robot from detecting external obstacles in its workspace, leading to collisions during limb movement.

Transitioning to Proactive Perception

Bridging the exteroceptive gap means moving from reaction to proactive planning. This requires integrating spatial data—like 2.5D height maps or raw visual streams—directly into the control policy. Wenhao Yu et al. demonstrate that for "discrete" terrains like stepping stones or large gaps, visual feedback is not an luxury but a requirement; proprioceptive-only policies fail almost immediately in these settings.

However, simply adding a camera is not enough. The challenge lies in the integration of these two distinct sensing modalities. Fu et al. proposed VP-Nav, a system that couples vision and proprioception so the robot can update its visual path planning based on what it physically feels, such as a slippery floor that "looks" safe but "feels" unstable.

Resilience and Sensor Degradation

The ultimate goal is a "full-stack" autonomy that remains resilient even when exteroception fails. In environments where cameras are blinded by dust, smoke, or low light, robots must fall back on their proprioceptive roots. Jin Jin et al. introduced a framework for "resilient navigation" that allows robots to "feel" their way through environments when sensors are compromised, maintaining high success rates even when completely blind.

Newer approaches like ProDapt attempt to give these blind systems a "memory" of the environment. By using diffusion models to remember past physical contacts and collisions, the authors enable robots to navigate complex obstacles using only touch, preventing the infinite loops and "tripping" that plague standard reactive controllers.

As researchers like Yecheng Jason Ma et al. automate the bridge between simulation and reality, the focus is shifting toward these multi-modal systems. The future of robotics lies in closing the gap between the internal sense of the body and the external perception of the world, allowing machines to move with the same proactive agility as biological organisms.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 253 papers
  2. Version 1 · 2026-10-04 · 351 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_exteroceptive_sensing_gap_7dcbfc9d,
  title = {exteroceptive sensing gap},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/exteroceptive-sensing-gap/rdr_7dcbfc9d90954fe2a3847af974f27bd8},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_exteroceptive_sensing_gap_7dcbfc9d_v2,
  title = {exteroceptive sensing gap},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/exteroceptive-sensing-gap/rdr_7dcbfc9d90954fe2a3847af974f27bd8/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake