Walking Past Blind Corners with a Stability-Aware Planner

A legged robot walking down a hallway cannot see what is around the next corner. If a person steps out, the robot usually has two ways to avoid them: stop, or step sideways. For a biped, stepping sideways is the dangerous one. Its side-to-side base of support is narrow, and unlike forward motion, momentum does not catch a sideways fall. Current systems split the robot's brain into a slow planner that decides where to go and a fast controller that keeps it upright. In that arrangement the planner does not know how far the body can safely lurch sideways right now, on this floor, at this speed. It walks toward blind corners at whatever pace it likes and trusts the fast controller to absorb any late emergency. This proposal reverses the direction of one message. The fast controller continuously reports a learned lateral escape envelope upward: how far it can safely move sideways within a given time without tipping. The planner then picks its speed and its distance from the wall so that a sideways escape stays physically possible against anyone who could emerge from the region it cannot see.

The two pieces of knowledge this needs are cheap to get, which matters in a field where robot data is scarce. The escape envelope is learned from simulated rollouts of the robot's own fast controller, with no demonstrations. The statistics of how fast and how close people emerge from blind spots are extracted from ordinary walking videos, which already contain many such events. The resulting claim is specific and falsifiable. A planner that plans against a state-dependent lateral envelope and a video-derived emergence prior should reach a better trade-off between travel time and collisions-plus-falls than any fixed speed limit near corners. The advantage should be largest when floor friction or slope changes, because that is where a single fixed limit is either too timid or unsafe.

Where the gap is

Hierarchical legged navigation already works. NaVILA has a vision-language model issue mid-level text commands such as "turn right 30 degrees" to a reinforcement-learned locomotion policy, and it trains the upper level on touring videos harvested from YouTube. The research file notes the two gaps that motivate this work. Its perception is purely egocentric. Its planner emits turning commands with no knowledge of the robot's roll margin, so the lower policy "must blindly absorb whatever rotational velocities are requested." More generally, the file names interface bottlenecks as an open problem for decoupled systems: language tokens and visual subgoals, as in SuSIE, strip out slip, contact, and momentum information. Our proposal is a narrow fix to that bottleneck. It adds one learned physical quantity to the interface and does not try to widen the whole channel.

Egocentric occlusion handling for legged robots also exists, and it constrains the design. World Model-based Perception uses a recurrent latent state to remember terrain that has left the camera view. Omni-Perception feeds raw 360° LiDAR into a near-field/far-field network trained end to end with reinforcement learning to dodge moving people. Neither can see through a wall: memory recovers only what was once visible, and a LiDAR cannot see past opaque occluders. We reuse their perception stacks. What we add is reasoning about the unobserved region, which neither system does.

Collaborative perception is the direct attack on occlusion. V2VNet, OPV2V, and CoBEVT fuse intermediate features from other vehicles or roadside units into a shared top-down map. It works when there are peers or infrastructure to share with, and indoor legged robots usually have neither. In our design, shared perception is not a competitor. Its only effect is to shrink the unobserved region the planner must guard against, and we test it as an upper-bound condition.

Lateral stability has been handled with formal safety filters. The Deep Koopman safety governor lifts vehicle dynamics into a linear latent space and uses control barrier functions (constraints that keep the state inside a safe set) to clip yaw rate and lateral velocity in a 0.1 ms quadratic program. Neural Gaits enforces barrier-function safety on a reduced walking model, but only on a planar, boom-constrained biped, and it names full 3D lateral balance as open. A safety filter of this kind stops the robot from tipping when a hazard appears. It cannot move the robot to a place where the hazard is avoidable. That is a planning-level property, and it is what we target.

On data, Learning to Drive Anywhere with Model-Based Reannotation relabels noisy trajectories using a 2D kinematic forward model. Humanoid-Gym shows that humanoid walking policies trained in massively parallel simulation transfer zero-shot. We rely on the second for the envelope. We borrow the video-mining approach of NaVILA for the emergence prior, but use it to extract hazard statistics, not instruction-action pairs.

Core Claim

Let $s$ be the robot's current proprioceptive and terrain state: body velocity, roll and pitch, and estimated friction and slope. Let $D_{\text{lat}}(s, t)$ be the largest sideways displacement the fast controller can achieve within time $t$ from state $s$ without falling. For each occlusion edge ahead, a hazard may emerge at distance $r$ from the robot's path while moving at speed $v_h$ drawn from a video-derived distribution. The robot, walking at forward speed $v$ with lateral offset $d$ from that edge, then has an available reaction time $t_c(v, d, r, v_h)$. That time comes from simple geometry: the time until the hazard's swept region meets the robot's path. The planner chooses $(v, d)$ to maximize progress subject to

$$ w(d, r) \le D_{\text{lat}}\big(s,\, t_c(v, d, r, v^{(q)}_h)\big), $$

where $w$ is the sideways clearance needed to avoid the hazard and $v^{(q)}_h$ is the $q$-th quantile of emergence speed. Equivalently, the planner may stop within $t_c$. The constraint then takes the better of lateral escape and braking, whose envelope $D_{\text{brake}}(s, t)$ is learned the same way. The claim is that two things beat any fixed rule: making $D_{\text{lat}}$ depend on state, and making $v_h$ come from data. The reason is that the safe approach speed legitimately varies several-fold with friction, slope, gait phase, and corner geometry.

Research Program

We first train a humanoid walking policy in the Humanoid-Gym setup, using its asymmetric actor-critic recipe, with velocity and sidestep commands and domain randomization over friction and slope. We then learn the envelope networks $D_{\text{lat}}$ and $D_{\text{brake}}$ by running the frozen policy from sampled states. From each state we issue an evasive command of a given magnitude and record the displacement achieved at each time and whether the robot fell. This is supervised regression on cheap simulated data. We fit a conservative quantile of achievable displacement rather than the mean, and we calibrate it on held-out rollouts so that the predicted envelope is violated less than a target rate. Sim-to-sim transfer to MuJoCo, following the Humanoid-Gym validation protocol, tests whether the envelope survives a physics change. A modest hardware calibration set would be a stretch goal.

In parallel, we mine egocentric walking videos, the same class of footage NaVILA uses, for emergence events: a person first appearing from behind a doorway or corner. Monocular metric pose estimation gives the distance at first appearance and the emergence speed. This yields an empirical distribution over $(r, v_h)$ conditioned on coarse context such as corridor or doorway. We expect this prior to be simple. The point is that it is measured rather than guessed, and we test whether it matters.

The planner sits between a goal-directed high level and the fast controller. It takes the occupancy map from the egocentric perception stack, extracts occlusion edges, queries the envelope at the current state, and solves the small constrained problem above at around 10 Hz. It passes the resulting speed and offset to the controller. It replaces NaVILA-style free-form turning commands only near occlusions, so the high-level goal interface is unchanged. When a shared top-down map from a peer or fixed camera is available, it simply removes occlusion edges.

Baselines

The naive reference is a hierarchical navigator in the NaVILA style (NaVILA). It issues goal-directed velocity commands to the same fast controller, has no occlusion reasoning, and relies on the controller's own reactive avoidance once a hazard is visible. It shows how bad the problem is.

The strongest reactive competitor is an end-to-end reinforcement-learning avoider in the style of Omni-Perception. We retrain it in our corridor scenes with pop-out pedestrians sampled from the same emergence prior, so it sees exactly the hazard distribution our planner assumes. It gets the same simulation budget as our controller plus envelope training combined, and equal hyperparameter search.

The strongest planning competitor, and the real threat, is a tuned fixed speed cap near occlusion edges plus a fixed minimum wall offset. We sweep its two parameters on a validation set of scenes and report its full Pareto curve, not a single point.

The stability-side competitor is a barrier-function safety filter on lateral velocity and roll rate. It is inspired by the Deep Koopman governor and fitted to the humanoid's commanded-velocity dynamics, and it is applied on top of the naive navigator.

Three ablations isolate the key ingredients. The first replaces $D_{\text{lat}}$ with its state-independent average, which tests state dependence. The second replaces the video prior with a hand-set worst-case emergence speed, which tests the data. The third removes the lateral term and keeps only braking, which tests whether the lateral direction matters at all. Finally, a full-visibility oracle, equivalent to perfect collaborative perception, bounds what removing occlusion could give.

Beating a baseline means that our method's Pareto curve of mean time-to-goal against combined collision-plus-fall rate lies below the baseline's curve across the test distribution. The target is at least a meaningful time reduction at matched safety on the low-friction and sloped splits, with confidence intervals over seeds and scenes. No reported result found on Lacuna for any of these baselines on an occluded-corner humanoid benchmark; the file contains no numeric figures for NaVILA, Omni-Perception, the Koopman governor, or Humanoid-Gym on such a task, so all numbers will be produced in-house under matched conditions.

Why not just…?

Why not just slow down near every corner? This is the most serious alternative, and the core experiment is designed around it. A fixed cap tuned for a dry, flat floor will produce falls or collisions on a low-friction or sloped floor, where the achievable sideways escape shrinks. A cap tuned for the worst floor wastes time everywhere else. The prediction is that one cap cannot sit on our Pareto curve across both conditions. If a single tuned cap matches us on every split, the state-dependent envelope is not earning its keep, and we will report that.

Why not just train an end-to-end policy on pop-out pedestrians? Omni-Perception shows that end-to-end reinforcement learning can dodge visible moving people. But the relevant decision, approaching slowly and wide before anything is visible, depends on corner geometry several meters ahead and on rare events, and sparse reward credits that poorly. The retrained avoider with matched compute directly tests whether it learns the anticipatory behavior anyway. If it does, the explicit envelope reduces to interpretability and calibration benefits, and we would say so.

Why not just put a lateral safety filter on the controller? A barrier-function filter like the Koopman governor prevents tipping during an evasive maneuver. It cannot create reaction time the robot does not have. We expect the filter baseline to trade falls for collisions, lowering fall rate while collision rate stays high. The per-outcome breakdown in our results will show whether that happens.

Why not just share perception with other agents? Where peers or fixed cameras exist, fused maps as in OPV2V remove occlusion and should beat any occlusion-aware planner. Our oracle condition measures that ceiling. But indoor legged deployments usually have no infrastructure, and the open problems listed for collaborative perception (latency, pose error, falsified feature injection) mean shared maps can drop out. Our planner treats shared views as optional evidence and falls back safely without them.

Proposed Experiments

The benchmark is a procedurally generated set of simulated corridors, doorways, and T-junctions built in the Humanoid-Gym environment. Pedestrians emerge from occluded regions with timing and speed sampled from the video prior. Floors are split into dry-flat, low-friction, and laterally sloped. Scenes are held out by layout. We report time-to-goal, collision rate, fall rate, and the minimum lateral stability margin (distance of the capture point, the spot where the robot would need to step to stop, from the edge of the support region), summarized as Pareto curves over 1,000+ episodes per split and several training seeds.

The decisive check is the friction/slope interaction. Our method should hold its safety level across splits by automatically slowing and widening on slippery floors, while the best fixed cap tuned on dry floors degrades. A second check is the braking-only ablation. If braking alone does as well as lateral escape for the humanoid, then lateral stability is not the binding constraint in this regime and the proposal's premise narrows accordingly. A third check is prior sensitivity: we replace the video-derived emergence distribution with a mismatched one at test time and measure how gracefully safety degrades. A fourth is sim-to-sim transfer of the envelope to MuJoCo, where we measure the envelope violation rate against its calibrated target.