Learning Sim-Trained Contact Models Through Invariant Filter Innovations

Legged robots estimate their own position and orientation by combining an inertial sensor with a simple rule: a foot touching the ground does not move. A small neural network decides which feet are in contact. Today that network is trained on labeled data from the real robot. Training it in a physics simulator would be far cheaper, but simulated contacts are too clean. They have no compliance, no micro-slip and no chatter. Earlier work collected real data for exactly this reason (Legged Robot State Estimation using Invariant Kalman Filtering and Learned Contact Events).

We propose to train the contact network in simulation and then adapt it on the real robot using only unlabeled logs. The adaptation signal is the filter's own prediction errors, called innovations: the gap between what the filter expected a sensor to read and what it actually read. The approach depends on one structural fact. In an invariant Kalman filter, a filter that represents pose errors as group elements on the space of rigid-body motions, the error dynamics do not depend on the trajectory the robot happens to walk. The statistics of the innovations should therefore reflect sensor and contact mismatch rather than gait or terrain path. If that holds, innovations are a clean, label-free measure of the simulation-to-reality gap, and the network can be tuned to make them statistically consistent.

Core Claim

A contact network trained in simulation outputs, for each foot, a contact probability and an input-dependent covariance for the "foot is stationary" measurement. It can be adapted to real hardware by maximizing the likelihood of the invariant filter's innovations on unlabeled real logs. We claim this works better inside an invariant extended Kalman filter (InEKF) than inside a standard quaternion EKF.

The reason is that InEKF innovations do not mix up two kinds of shift. One is the shift we want to correct: real feet slip and simulated feet do not. The other is a nuisance: the real robot walks different trajectories than the simulated one. A quaternion EKF linearizes around the estimated state, so its innovation statistics change with the trajectory even when sensor noise is identical. That turns the same objective into a noisy, trajectory-dependent target.

The adaptation objective is

$$ \mathcal{L}(\theta) = \sum_t \left[ \nu_t^\top S_t(\theta)^{-1} \nu_t + \log\det S_t(\theta) \right], $$

where $\theta$ are the network weights, $\nu_t$ is the contact-measurement innovation at time $t$ expressed in the filter's right-invariant error coordinates, and $S_t(\theta)$ is its predicted covariance. That covariance is the propagated state uncertainty plus the network's predicted measurement covariance, weighted by its contact probability. This is the standard Gaussian innovation likelihood. Its two terms penalize, respectively, overconfident covariances and covariances inflated to hide errors. Pure consistency targets that only push whitened innovations toward unit variance lack the second term, so they reward inflation. The objective needs no ground-truth contact labels and no motion capture.

Relation to Prior Work

Three papers motivate the proposal.

  • Learning Locomotion Skills for Cassie: Iterative Design and Sim-to-Real showed that a control policy transfers from simulation when it is trained on the outputs of the robot's noisy state estimator, which the authors call estimator parity. They treated that estimator as a fixed black box. We make its learned part the object of transfer.
  • Lin et al. (cited above) feed a convolutional network over 150-step windows of joint and inertial signals into an InEKF and reach 97.82% contact accuracy on the Mini Cheetah. They train on real data, output only binary contacts, and use hand-tuned fixed covariances. We keep their architecture but add a covariance output, train it in simulation, and adapt it without labels.
  • Learning Inertial Odometry for Dynamic Legged Robot State Estimation shows that learned components work best inside a filter. Their EKF-IKD reported a 37% lower relative pose error on slippery and soft ground.

Four papers supply the method.

Three papers constrain the design.

Research Program

The work has three parts.

Simulation training. We build a simulated Mini Cheetah (and later Cassie) pipeline that logs joint encoders and inertial readings. We add slip and compliance randomization at several strengths. The contact network is trained in simulation with a loss computed through the InEKF update, so that contact probabilities and covariances are trained to minimize filter error, not just classification loss. This is estimator parity applied to the estimator itself.

Unlabeled real adaptation. We adapt only the covariance head and the final layers by minimizing the innovation likelihood on real logs. We compare three filter cores sharing that network:

  • a quaternion multiplicative EKF,
  • the InEKF with bias states,
  • a TG-EqF.

Measuring the premise. In simulation, where the truth is known, we hold sensor noise fixed and shift only the trajectory distribution (gait, speed, turning rate). We then quantify how much each filter's innovation covariance moves, using the log-Euclidean distance. This tells us directly whether InEKF innovations are trajectory-independent in practice once biases and slip are present. It also bounds how much of the method's benefit can come from invariance.

Baselines

Do-nothing reference. The simulation-trained network deployed unchanged inside the InEKF, with no adaptation. Every adaptive method must beat it on real absolute trajectory error. If it is already close to the real-data model, the gap we target is small and the project's premise is weak.

Real-data upper bound. Lin et al.'s network retrained on real hardware with their self-supervised contact labels and fixed covariances. Their reported number is 97.82% average contact accuracy across eight terrains. We rerun it on the same splits because their paper reports contact accuracy, not trajectory error. We do not expect to beat it on accuracy. Success is closing at least half the trajectory-error gap between the do-nothing reference and this bound while using no real labels. We will also report whether the learned covariance improves on the bound's own fixed covariances, measured by trajectory error and by filter consistency.

Strongest learned-in-filter method. EKF-IKD from the legged inertial odometry paper, with its reported 37% relative pose error reduction on slippery and soft ground. We will reimplement its learned displacement factor, train it in simulation, and give it the same unlabeled real logs and the same adaptation budget.

Distance-weighting competitor. The down-weighting rule from Reichlin et al.: drop measurements that land far from the motion prediction. Here it replaces the learned covariance. No reported result found on Lacuna for it on legged state estimation.

Key ablation. The identical network and adaptation objective inside the quaternion EKF. This isolates whether invariance is the active ingredient.

Fairness. All methods receive the same simulation data, the same unlabeled real hours, the same hyperparameter search budget tuned on a held-out real terrain, and the same embedded compute target. Lin et al. report 830 Hz on a Jetson AGX Xavier, which sets that target.

Why not just…?

Why not just tune a few covariance scalars on real innovations, classical adaptive-filter style? This is the most serious threat. If slip on real terrain is roughly uniform, a global per-foot noise scale fitted by the same likelihood could recover most of the gain. A context-dependent covariance head would then be unnecessary. We run exactly this as a baseline, with the identical objective and data. If it matches the learned head within noise on all eight terrains, we would report that and narrow the claim to "invariant innovations make unlabeled covariance tuning reliable." We expect the learned head to win specifically on mixed terrain such as pebbles and forest debris, where slip varies step to step.

Why not just collect real labels as Lin et al. did? On one robot and terrain set, that works very well, at 97.82%, and it is our upper bound. The case for simulation is the cost of re-collecting labeled data for each new robot, terrain, or sensor change. The Cassie generalization experiment tests whether simulation training plus unlabeled adaptation reaches a new platform faster than relabeling does.

Why not just randomize slip heavily in simulation and skip adaptation? Xie et al. found that randomization was unnecessary once estimator parity and accurate system identification were in place. This suggests that targeted correction can beat brute-force randomization, but it does not settle the question for contact estimation. We sweep randomization strength in the do-nothing reference. If strong randomization alone closes the gap, adaptation is unnecessary and we will say so.

Why not just use an end-to-end network? The legged inertial odometry and NeuroKalman results both show that purely learned dead reckoning drifts without bound. We therefore do not expect an end-to-end estimator to compete on long walks. We still include one, trained in simulation with randomization, as a sanity check.

Decisive Checks

Real-hardware evaluation. We use the Mini Cheetah eight-terrain data from Lin et al., with motion capture or loop-closure reference trajectories collected only for evaluation. We report:

  • absolute trajectory error,
  • relative pose error,
  • drift per meter,
  • average normalized estimation error squared (ANEES), a filter-consistency score that should be near 1 for a well-calibrated filter.

We train on six terrains and test on two held-out terrains. If the ANYmal mine logs from the legged inertial odometry paper are obtainable, we evaluate there as well.

Three decisive outcomes. - Does the InEKF-plus-adaptation recover at least half of the gap to the real-label bound? - Does the same objective inside the quaternion EKF recover clearly less? - Does the simulation trajectory-shift study show smaller innovation-covariance movement for the InEKF and TG-EqF than for the quaternion EKF?

If the second or third outcome fails, the invariance argument fails. What would remain is a generic unlabeled calibration method, and we would report it as such.