A wheeled robot's sensors often disagree. Wheel encoders, an inertial measurement unit (IMU, an accelerometer plus gyroscope) and a camera can give different answers about how the robot moved. Current estimators treat that disagreement as noise to be down-weighted or as outliers to be thrown away. We propose to treat it as a clue about its cause. When the wheels say the robot moved farther than the IMU and camera do, the most likely cause is that the tires slipped. A slip event says something useful about the road: at that moment, the robot's horizontal acceleration hit the limit the surface could support. Every moment without slip carries a weaker but still useful statement: the surface supported at least the acceleration the robot was undergoing.
We will build an estimator that infers, at each instant, whether the wheels are gripping or sliding. It then turns that inference into a running estimate of road friction, and uses the friction estimate to limit how the robot's motion is predicted when the camera goes dark. In two sentences: model wheel-ground slip with a prior that says "usually exactly zero, occasionally anything," so disagreements between sensors are attributed to slip rather than noise. Then reuse the resulting grip and slip episodes to bound road friction, and propagate the vehicle state through visual blackouts using motions that friction actually permits.
Core Claim
The prior is borrowed from perception science. Hierarchical motion perception as causal inference models how humans group moving dots. It assigns relative velocity a mixture prior: a spike at exactly zero, standing for things in contact staying put relative to each other, plus a broad slow-speed Gaussian. That model explains 92–96% of the variance in human reports. Combination and competition between path integration and landmark navigation shows the same structure in heading estimation, where people average body-based and visual cues when they roughly agree and let one override the other when they clearly disagree. Both papers are perceptual and use synthetic stimuli or VR. Neither touches a vehicle. We take the mathematical form, not the biology. This is the method this work supplies.
Concretely, let $s_t$ be the velocity of the tire contact patch relative to the ground at time $t$, estimated from the gap between wheel-encoder speed and the chassis velocity implied by the IMU and the camera. We place on it the prior
$$p(s_t) = \pi\,\delta(s_t) + (1-\pi)\,\mathcal{N}(s_t;\,0,\,\sigma_s^2),$$
where $\pi$ is the prior probability of grip, $\delta$ is a point mass at zero, and $\sigma_s$ is the typical slip speed. The posterior grip probability $g_t = p(s_t = 0 \mid \text{data})$ decides, softly, whether wheel odometry constrains the pose.
The second half of the claim is what makes this more than robust weighting. Let $a_t$ be the measured horizontal acceleration and $\gravity$ gravity's acceleration $9.81$ m/s². Grip episodes give a censored observation and slip onsets give a point observation of the friction coefficient $\mu$:
$$\mu \ge \frac{\lVert a_t \rVert}{\gravity} \ \text{ when gripping}, \qquad \mu \approx \frac{\lVert a_t \rVert}{\gravity} \ \text{ at slip onset}.$$
This follows from the friction-circle limit: the tire can transmit at most $\mu$ times the normal load in any horizontal direction. We maintain a posterior over $\mu$ with slow forgetting, so the estimate tracks surface changes, using the weights $g_t$ to mix the two observation types. During a camera blackout, the propagated motion must stay inside the circle $\lVert a \rVert \le \mu \gravity$, with $\mu$ drawn from that posterior.
The novelty is that slip goes from being a nuisance that existing estimators discount to being the measurement that identifies friction. That friction estimate in turn constrains prediction exactly when exteroceptive sensing is lost.
Relation to Prior Work
Three systems sit closest, and all three discount slip instead of exploiting it.
WING trains networks that output corrected wheel and IMU measurements with learned uncertainty, so encoders are discounted during slip, and it constrains pose to a smooth ground surface. It reaches 8.10 m absolute trajectory error on KAIST urban07, against 30.99 m for a plain extended Kalman filter. WING uses no camera and produces no friction estimate. Its learned covariance is a black box rather than an explicit grip/slip variable.
The fuzzy wheel-inertial-visual filter adapts noise levels from wheel-speed discrepancies and reprojection errors using hand-written fuzzy rules. It has been validated only at low speed indoors.
VIMO is the closest in spirit. It adds a dynamics factor and an external-force state to visual-inertial odometry, which cuts error 29% versus VINS-Mono on quadrotors. It has no tire contact, slip or friction.
We keep VIMO's idea that the estimator should carry physical quantities as states. We replace WING's and the fuzzy filter's down-weighting with explicit causal attribution.
Two lines of evidence motivate the work. ROOAD reports that OpenVINS and VINS-Fusion perform 2 to 30 times worse off-road than on EuRoC, which shows that rigid-body visual-inertial assumptions break on loose terrain. Lateral Trajectory Following at Handling Limits shows that friction back-calculation diverges when ground velocity is poorly estimated. Our design has to escape this circularity: friction is computed from velocity that is itself corrupted by slip. Here the camera breaks the loop when it is available, and the friction posterior simply freezes, with growing uncertainty, when it is not.
Existing online friction estimators supply useful machinery but rest on different assumptions. The nonlinear moving-horizon estimator on TerraSentia and forgetting-factor least squares for cornering stiffness in the coordinated stability controller assume continuous, synchronous wheel and inertial data and a reliable velocity reference. We reuse the forgetting idea but swap the regression for a censored update that works from sparse slip events.
For prediction through blackouts, PIVO shows that carefully maintained covariance lets an inertial-visual filter dead-reckon through a full camera occlusion on a phone, with 0.23% end error over 110 m. It has no vehicle model. MODE and Bridging Time and Frequency show that low-rank neural ODEs handle irregularly sampled multivariate series efficiently (a neural ODE is a network that learns a continuous-time rate of change). They have been tested only on tabular benchmarks such as ETT and Weather. We use such a model only as an optional learned residual inside the friction bound, and we test whether it earns its place.
MambaTrack, a learned state-space tracker that bridges detection gaps autoregressively in image coordinates, motivates a later extension rather than the core. Image-space trackers cannot separate camera skid from target motion, and a grip/slip ego estimate is a prerequisite for fixing that.
Research Program
The estimator is an error-state Kalman filter over pose, velocity, IMU biases, a per-axle slip velocity, and a friction posterior. The spike-and-slab slip prior is handled as a two-hypothesis mixture at each wheel update: we run grip and slip branches and merge them by posterior weight. This keeps the cost a small constant factor over a standard filter, so it should run on embedded hardware, unlike full moving-horizon estimation. The camera enters as ordinary visual-odometry updates. When the camera is unavailable, slip attribution relies on IMU-versus-wheel disagreement alone, with correspondingly lower confidence.
Friction is a scalar with a skewed posterior. It is lower-bounded by grip episodes, pinned by slip onsets, and inflated over time by a forgetting factor. During blackouts, propagation uses a kinematic bicycle model whose commanded accelerations are clipped to the friction circle at a chosen posterior quantile. The optional variant adds a low-rank neural ODE residual, trained on logged data, whose output is projected back inside the circle.
We will analyze three things. The first is identifiability: how much acceleration excitation is needed before the lower bound becomes informative. On gentle urban driving it may never approach the true $\mu$, and we will report that honestly. The second is failure under simultaneous slip and blackout. The third is calibration of $g_t$ against labeled slip events.
Data comes from three sources. KAIST Urban provides wheel encoders, IMU and ground truth for comparison with WING; we will use camera streams if the release provides usable ones. NCLT provides cross-domain transfer. Both are mostly high-grip driving. ROOAD provides off-road camera-IMU data but, per the file, no wheel encoders, so there it tests only the camera-IMU attribution path. Friction ground truth exists in none of these, so we add an instrumented 1/10-scale car, like the platform in Consecutive Inertia Drift of Autonomous RC Car. It runs under motion capture on surfaces whose $\mu$ we measure by drag tests: dry board, polished floor, a low-friction sheet, and loose gravel. Each surface gets scripted acceleration and cornering maneuvers, and we inject camera blackouts of 2–20 s.
Baselines
The do-nothing reference is a standard error-state filter that fuses wheel, IMU and camera with fixed noise and no slip model. During blackouts it propagates with constant velocity. On KAIST urban07, the analogous wheel-inertial filter reported in WING scored 30.99 m absolute trajectory error (WING).
The strongest existing method for ego-pose under slip is WING, which reports 8.10 m on KAIST urban07 and 8.22 m zero-shot on NCLT 13/04/05. We will run its released configuration if available; otherwise we will reimplement it and train on the same KAIST splits it reports. Our method gets no more training data. Because WING has no camera, the fair comparison is our system with the camera stream disabled. Beating WING means matching or improving on 8.10 m without learned covariance networks. We will separately report the gain from adding the camera.
For camera-IMU pipelines off-road, the references are OpenVINS and VINS-Fusion on ROOAD. ROOAD reports these as 2–30 times worse than on EuRoC; the file gives no per-sequence numbers, so we will run them ourselves with identical calibration. For blackout propagation, the reference is PIVO-style inertial dead reckoning with full covariance. Its only reported vehicle-free figure is 0.23% over 110 m on a phone. No reported result was found on Lacuna for PIVO on wheeled platforms, so we will run a reimplementation.
For friction, no reported result was found on Lacuna for online $\mu$ error on any of these datasets. Our baseline will be forgetting-factor least squares on wheel-inertial data, in the style of the coordinated stability controller, given the same signals and tuned on the same held-out runs.
Two ablations isolate the key ingredients. Replacing the spike-and-slab prior with a single Gaussian of matched variance tests whether causal attribution matters. Dropping the friction-circle clip during blackouts, while keeping everything else, tests whether the friction estimate is what helps prediction. All methods get the same hyperparameter search budget on a shared validation split.
Why not just…?
Why not just gate wheel odometry with a chi-square outlier test? This is the real threat for pose accuracy. Chi-square gating also makes a hard grip-or-slip decision, and it may match our trajectory error on KAIST. What gating cannot give is a friction estimate, because it discards the slip residual rather than estimating it, and it carries no notion that grip episodes bound $\mu$ from below. The Gaussian-prior ablation, plus a gated variant we will add, settles this. If gating matches us on pose and our advantage appears only in friction error and blackout drift on the instrumented car, we will state the contribution exactly that way.
Why not just learn measurement uncertainty end to end, as WING does? WING's 8.10 m result shows learned covariance works well for pose. Its outputs are not a physical quantity, though, and they cannot be reused to bound prediction when sensors go dark. We compare head-to-head on KAIST with the camera off. If WING wins on pose, the useful hybrid is to use its learned covariance inside our slip branch, and we will test that.
Why not just forecast through the blackout with a learned sequence model such as MODE? MODE handles irregular sampling efficiently, but it has never been evaluated on vehicle states. Nothing in an unconstrained rollout stops it from predicting cornering accelerations a slick floor cannot support. The clip-versus-no-clip ablation on the low-friction surfaces measures exactly this. If the learned residual alone stays within the friction circle and matches our drift, the friction bound is unnecessary for prediction, and the paper's weight shifts to estimation.
Why not just integrate the IMU through the blackout? ROOAD's 2–30 times degradation off-road and VIMO's 29% gain from a dynamics model both suggest that pure inertial integration is weak on rough ground. The PIVO-style baseline measures how weak on our platform.
Proposed Experiments
The decisive check is on the instrumented car. We start the robot on a high-friction surface, drive it onto the low-friction sheet, and black out the camera at the transition. A method that has seen a slip onset on the sheet should predict wider, slower turns than one that kept its high-grip belief. Here we expect the friction-bounded propagation to cut end-of-blackout position error against both constant-velocity and unconstrained-learned propagation. We expect the effect to vanish on uniform high-grip surfaces. Seeing the effect only where friction changes, and not elsewhere, is the signature that the mechanism is causal rather than incidental.
Secondary checks are friction estimation error and convergence time per surface against the least-squares baseline. Grip-probability calibration is scored as Brier score against motion-capture slip labels, where slip is defined as contact-patch speed above a threshold. We also report absolute trajectory error on KAIST and NCLT against WING's 8.10 m and 8.22 m.
We also report a negative case in advance. On KAIST-style gentle driving, the lower bound on $\mu$ should rarely tighten, so blackout gains there should be near zero. If they are not near zero, something other than the friction mechanism is responsible, and we will investigate before claiming the result.
References
- Hierarchical motion perception as causal inference
- Combination and competition between path integration and landmark navigation in the estimation of heading direction
- WING: Wheel-Inertial Neural Odometry With Ground Manifold Constraints
- Multi-Sensor Fusion for Wheel-Inertial-Visual Systems Using a Fuzzification-Assisted Iterated Error State Kalman Filter
- VIMO: Simultaneous Visual Inertial Model-Based Odometry and Force Estimation
- ROOAD: RELLIS Off-road Odometry Analysis Dataset
- Lateral Trajectory Following for Automated Vehicles at Handling Limits
- Embedded High Precision Control and Corn Stand Counting Algorithms for an Ultra-Compact 3D Printed Field Robot
- Coordinated Control Strategy for Stability Control and Trajectory Tracking with Wheel-Driven Autonomous Vehicles Under Harsh Situations
- PIVO: Probabilistic Inertial-Visual Odometry for Occlusion-Robust Navigation
- MODE: Efficient Time Series Prediction with Mamba Enhanced by Low-Rank Neural ODEs
- Bridging Time and Frequency: A Joint Modeling Framework for Irregular Multivariate Time Series Forecasting
- MambaTrack: A Simple Baseline for Multiple Object Tracking with State Space Model
- Consecutive Inertia Drift of Autonomous RC Car via Primitive-Based Planning and Data-Driven Control