Traditional robot control often relies on "seeing" the world through cameras or LiDAR. However, vision can be unreliable: cameras get obscured by mud, lighting changes, or the robot’s own chaotic movements during a fall. A growing body of research suggests that robots can instead learn to "feel" their environment by treating their own bodies as sensors. By analyzing the history of their own joint movements and motor commands, robots can implicitly infer external physical properties—like surface friction, terrain height, or object mass—and adapt their behavior in real-time.
The Body as a Sensor
The core intuition behind this research direction is that every interaction with the environment leaves a "signature" in the robot's proprioceptive data. If a quadruped robot steps on ice, its feet will slip, creating a specific discrepancy between the commanded motor torque and the resulting joint velocity. If a robotic hand tries to rotate a heavy object, the fingers will lag behind their target positions differently than they would for a light object.
Instead of manually modeling these complex physics, researchers like Ashish Kumar, Deepak Pathak, and Jitendra Malik proposed Rapid Motor Adaptation (RMA). In RMA: Rapid Motor Adaptation for Legged Robots, the system uses a small neural network—an adaptation module—to look at a short history of joint states. This module learns to output a "latent code" that represents environmental conditions like ground friction or payload weight, allowing the robot to adjust its gait within fractions of a second.
Inferring Latent Context from History
Why does a temporal history matter? A single snapshot of joint angles tells you where the robot is, but it doesn't tell you why it is there. By looking at a window of time, the system can see the relationship between cause (motor commands) and effect (actual movement).
In the context of manipulation, Haozhi Qi and colleagues applied this to in-hand rotation in In-Hand Object Rotation via Rapid Motor Adaptation. By monitoring the discrepancy between commanded actions and actual joint movements, the robot hand can "feel" the shape and friction of an object without any tactile or visual sensors. This allows it to rotate diverse objects—from shuttlecocks to marbles—using the same adaptive controller.
This concept extends to terrain geometry as well. I Made Aswin Nahrendra and others developed DreamRiser, which uses "implicit terrain imagination" to help robots recover from falls. As described in Robust Recovery Motion Control for Quadrupedal Robots via Learned Terrain Imagination, the robot senses the slope and roughness of the ground through its joint movements, allowing it to "imagine" the terrain it is lying on and choose the right way to stand back up.
Internal Models and Disturbance Estimation
A more recent evolution of this idea treats environmental factors as "disturbances" to an internal model of the robot. In Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response, Junfeng Long and Zirui Wang show that by modeling how the robot's body should respond in a vacuum, any deviation in the real world can be mapped directly to terrain properties like height or friction.
This "internal model" approach is particularly powerful for complex hardwares:
- Wheel-Legged Robots: Xinghua Yu and the Tencent Robotics X team demonstrated in Whole-Body Impedance Coordinative Control of Wheel-Legged Robot on Uncertain Terrain that analyzing wheel trajectories and contact forces allows a robot to identify slopes and irregularities in real-time, maintaining stability even when carrying a tray of water.
- Humanoids: Junfeng Long further extended this to humanoids in Learning Humanoid Locomotion with Perceptive Internal Model, where terrain data is integrated into state estimation to help the robot distinguish between external resistance and its own motor performance.
Knowing What You Don't Know
While adaptation is powerful, it can be dangerous if the robot encounters a situation completely unlike its training data. James Queeney and Jonathan P. How addressed this in GRAM: Generalization in Deep RL with a Robust Adaptation Module. Their framework, GRAM, uses an epistemic neural network to detect "out-of-distribution" environments. When the robot's proprioceptive history indicates a situation it doesn't recognize, it automatically switches from a high-performance agile mode to a conservative "safe mode," ensuring stability even when adaptation fails.
By shifting the burden of environmental sensing from external vision to internal proprioception, these methods have made robots significantly more robust. They no longer need a perfect map of the world; they simply need to be able to feel the ground beneath their feet.