A Risk Dial at the Footstep Level: Noise-Aware Humanoid Stepping with Reduced-Order Models

Walking humanoids are usually controlled in layers. A slow planner decides where the feet should land using a simple "template" model of the robot, such as an inverted pendulum. Fast controllers below it then make the full body follow that plan. The planner almost always treats the robot's estimated state as exact. On hardware the estimate is not exact. The inertial sensor has bias, the joint encoders are noisy, the contact sensors sometimes report a foot as planted when it is slipping, and the estimator's velocity drifts. A footstep that looks safe for the estimated state can be a fall for the true state.

We propose to make the footstep planner explicitly risk-averse to this estimation error, with a single continuous knob that a supervisor can turn at run time without retraining. The knob sets how much weight the planner puts on bad outcomes. The idea rests on one observation. The template models used for stepping have step-to-step dynamics that are linear within each contact mode. Under a small set of discrete "what went wrong" modes (nominal contact, slip, missed touchdown), the spread of next-step outcomes is therefore a Gaussian mixture that can be computed cheaply. The tail-risk measure (conditional value-at-risk, the average cost over the worst fraction of outcomes) then costs roughly a one-dimensional root-find, not thousands of samples. This makes tunable risk aversion cheap exactly where it matters most, in foot placement. It also avoids the sampling problems that make tail estimates unreliable in sampling-based controllers.

Core Claim

We claim that placing the tunable risk measure at the reduced-order footstep layer is better than placing it in the low-level learned policy or ignoring it. The measure is computed over an online estimate of state uncertainty that includes discrete mode switches. It should give a humanoid a smooth, monotone trade between speed and fall rate as the knob turns. It should also transfer across a train-to-test shift in sensor noise better than a risk-conditioned reinforcement-learning policy whose risk estimates are tied to its training noise.

The mixture over modes is what separates this from simply widening safety margins. When uncertainty is a single Gaussian, the tail risk reduces to "mean plus a constant times the standard deviation." That is ordinary margin inflation. We treat that case as a baseline the method must beat, not as the method itself.

Grounding in Prior Work

Several papers supply the machinery. Centroidal and template planning is established. Kuindersma et al. plan over centroidal momentum on Atlas (Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot). A hybrid pendulum model guides contact-implicit predictive control and raises push-recovery success from 30% to 62% (Reduced-Order Model Guided Contact-Implicit Model Predictive Control for Humanoid Locomotion). Ghansah, Esteban and Ames run a three-tier stack on a Unitree G1 (Hierarchical Reduced-Order Model Predictive Control for Robust Locomotion on Humanoid Robots):

  • a 40 Hz planner using an angular-momentum pendulum model (ALIP) and its step-to-step map;
  • a 500 Hz rigid-body predictive controller;
  • a command mapper.

That last stack is our host architecture. All three plan against the estimated state as if it were true, and none exposes a risk parameter. We keep their layers and change only what the top layer optimizes and what it is told about uncertainty.

The risk side supplies the method for the objective. Sopasakis et al. show that risk-averse predictive control for systems that switch between Markov modes can be written as a tractable quadratically constrained program over scenario trees (Risk-averse model predictive control). We reuse this structure, but apply it to switches driven by estimation and contact errors in legged stepping. Their work targets generic switching systems, not locomotion.

The case against sampling at the tails comes from risk-aware path-integral control. Yin et al. cut collisions by 80% in simulation using conditional value-at-risk over disturbance samples (Risk-Aware Model Predictive Path Integral Control Using Conditional Value-at-Risk). The same work, together with RAP: Risk-Aware Prediction for Robust Planning, notes that the effective sample size collapses as risk aversion grows. That limitation motivates our semi-analytic computation.

On the learning side, Schneider et al. train a quadruped policy conditioned on a risk parameter using a distributional critic, a value estimate that predicts the whole distribution of returns rather than its average (Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learning). At its risk-averse setting, the policy refused a 41 cm step 85.7% of the time. The GRIPS method generalizes this across risk measures but is validated only in simulation (Risk-Conditioned Reinforcement Learning: A Generalized Approach for Adapting to Varying Risk Measures). These are our main competitors. In both, risk lives inside a learned critic whose return distribution reflects the training noise. Our approach puts risk in an explicit model that reads the current noise estimate.

Two gaps in the file itself shape the design. The hierarchy literature warns that interfaces which discard physical state leave the planner blind to infeasibility; NaVILA's text commands are the file's example (NaVILA: Legged Robot Vision-Language-Action Model for Navigation). Passing a risk level and an uncertainty estimate across the layer boundary is one small remedy. The file also contained no papers or benchmarks specific to sensor-noise sim-to-real transfer. We therefore build the noise protocol ourselves. We report no prior numbers for it because there is no reported result found on Lacuna for humanoid stepping under controlled sensor-noise shift.

Research Program

We will build three components. The first is an uncertainty-aware version of the ALIP step-to-step map. At each step it propagates two things: the estimator's covariance over the center-of-mass state and angular momentum, and a small set of contact modes. The modes are nominal, stance-foot slip modeled as a velocity jump, and late or missed touchdown modeled as altered step timing. Mode probabilities come from contact-detection confidence and slip indicators.

Within each mode the map is linear, so the next-step state is a mixture of Gaussians. The planner chooses step location and timing to minimize

$$J_\alpha(u) = \mathbb{E}[c(x_{k+1})] + \lambda\,\mathrm{CVaR}_\alpha[c(x_{k+1})],$$

where the symbols are defined as follows:

  • $u$ is the step location and duration;
  • $x_{k+1}$ is the next-step template state;
  • $c$ is a quadratic cost on deviation from a capturable (recoverable) state plus a barrier on kinematic reach;
  • $\alpha \in (0,1)$ is the tail fraction, which is the operator's knob;
  • $\lambda$ is a fixed weight.

For a quadratic cost over a Gaussian mixture, the conditional value-at-risk reduces to a one-dimensional search for the value-at-risk threshold plus closed-form partial expectations. This keeps the 40 Hz budget.

The second component is the covariance feed. The state estimator's covariance and the contact confidences go to the planner every cycle. The planner therefore becomes more cautious when the sensors degrade, even at a fixed $\alpha$.

The third component, used only as an ablation, passes $\alpha$ down to a risk-conditioned low-level policy trained in the style of Schneider et al. This tests whether consistency across layers adds anything beyond top-level risk.

On the analysis side, we will establish two things for the linear-mixture case. One is when the optimum of $J_\alpha$ changes monotonically in $\alpha$, which is what makes the knob interpretable. The other is how large the gap between the mixture risk and the single-Gaussian margin is as a function of slip probability. The second result predicts when our method should differ from margin inflation at all.

Baselines

The naive reference point is the Ghansah et al. G1 stack run exactly as published, with a risk-neutral ALIP planner fed only the state estimate. It is the correct "do nothing" because it is the architecture we modify. The file reports no noise-robustness number for it, so we reproduce it in our own simulator under our noise protocol.

The strongest classical competitor is the same stack with covariance-proportional margin inflation, in the spirit of chance constraints. Its margin multiplier is swept over the same number of settings as our $\alpha$ and it receives the same online covariance. It is the most dangerous baseline because, under single-Gaussian noise, it is mathematically equivalent to our method. We will count a win only if our method strictly dominates its speed-versus-fall Pareto curve on the slip-heavy and contact-misdetection conditions. We also expect parity on pure Gaussian encoder noise, and will report it.

The strongest learning competitor is a risk-conditioned distributional policy following Schneider et al. It is trained end to end on the G1 in Isaac Sim with domain-randomized sensor noise, using the same total simulation budget as our low-level controller training plus equal hyperparameter-tuning trials. For reference, the file reports that at its risk-averse setting the published method refused a 41 cm step 85.7% of the time on ANYmal, versus 0% refusals (100% attempts) for its risk-neutral and risk-seeking settings. That is a behavior-selectivity number, not a fall rate, so we use it only to check that our reimplementation produces comparable selectivity on an analogous obstacle. Beating this baseline means a lower fall rate at matched forward speed under the held-out noise distribution.

A secondary comparison point is the hybrid-pendulum-guided contact-implicit controller, with its reported push-recovery success of 62% (Reduced-Order Model Guided Contact-Implicit Model Predictive Control for Humanoid Locomotion). Its robot and push protocol differ from ours, so we use it only as a calibration anchor for our push protocol, not as a head-to-head number.

Four ablations isolate the key ingredients:

  • a single-Gaussian version of our planner, with modes removed;
  • a fixed-covariance version, with the online feed removed;
  • a version with risk only at the low level;
  • a version with risk at both levels.

Why not just…?

Why not just randomize sensor noise during training and skip explicit risk? Domain randomization makes a policy robust on average to the training noise range, but it gives the operator no knob. It also cannot react when noise at test time leaves that range. Our held-out noise condition tests this directly. We include a randomized risk-neutral policy as a point on the comparison plot. If it matches our fall rate at matched speed under the shifted noise, the explicit risk layer is not earning its keep.

Why not just inflate footstep margins by the estimator's covariance? This is the real threat, and we do not argue around it. For Gaussian errors it is identical to our objective. Our claim is limited to cases where contact-mode errors make the outcome distribution bimodal. There a single margin is either too timid in the nominal mode or blind to the slip mode. The single-Gaussian ablation and the tuned-margin baseline settle this. If neither loses on the slip conditions, we will report that the mixture modeling is unnecessary.

Why not just use a risk-conditioned reinforcement-learning policy end to end, as for ANYmal? That approach works on hardware (Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learning). However, its tail estimates come from a learned critic trained under one noise distribution. Quantile critics are also known to need extra machinery to stay well-behaved (Risk-Conditioned Reinforcement Learning: A Generalized Approach for Adapting to Varying Risk Measures). The noise-shift experiment tests whether a model that reads live covariance degrades less.

Why not just sample the tail, as in risk-aware path-integral control? Sampling works at moderate risk levels. The effective sample size, however, collapses as $\alpha$ shrinks (Risk-Aware Model Predictive Path Integral Control Using Conditional Value-at-Risk). We will include a sampled version of our own objective and plot estimate variance and compute time against $\alpha$.

Proposed Experiments

The main testbed is a simulated Unitree G1 running the Ghansah et al. hierarchy, in both MuJoCo and Isaac Sim. The noise protocol has a training distribution of encoder noise, inertial-sensor bias, and estimator drift. It also has a held-out distribution with larger bias, heavier-tailed encoder noise, and injected contact misdetections and stance slips at controlled rates.

The tasks all make foot placement matter:

  • flat walking with lateral and sagittal pushes;
  • stepping stones with gaps;
  • a narrow beam;
  • a low-friction patch.

The metrics are fall rate, the conditional value-at-risk of capture-point error at touchdown, mean forward speed, and planner compute time. The central plot is fall rate against speed as $\alpha$ sweeps, one curve per method. We want to see whether the curve is monotone and whether it dominates the others.

Three results decide the question. The first is whether our curve dominates the tuned-margin baseline on slip and misdetection conditions while matching it on Gaussian-only noise; that result is what the analysis predicts. The second is whether the gap to the risk-conditioned learning policy widens on the held-out noise, relative to the training noise. The third is whether $\alpha$ changes behavior monotonically: fewer falls and shorter, more frequent steps as caution rises. A non-monotone knob would make the method unusable by an operator, whatever its average numbers.

If a G1 is available, a limited hardware check on push recovery and a low-friction mat would test only one thing: whether the knob's ordering survives transfer to real hardware. It would not attempt to reproduce the simulation numbers.