Speed-Conditioned Tail-Risk Terrain Queries for Tunable Off-Road Planning

An off-road vehicle's risk from a patch of ground depends on how fast it crosses it. A bump that is harmless at walking pace can crack a suspension at 10 m/s. Current learned terrain models capture only one side of this. Some predict how rough a patch will feel at a given speed, but they predict only the average. Others predict how bad the worst outcomes on a patch could be, but they ignore speed.

We propose one small network that answers a single question, many thousands of times per planning cycle: "if the vehicle crosses this location at this speed and heading, how bad is the worst α-fraction of outcomes?" Here α is a dial the operator can turn at runtime without retraining. The network is queried only at the states a sampling-based controller actually imagines visiting. That controller samples many candidate futures and blends the good ones. The worst-case cost it returns replaces both the average-cost terrain penalty and the expensive nested simulation that current tail-risk controllers use.

The claim worth testing is narrow and falsifiable. Off-road outcome distributions become heavy-tailed in a speed-dependent way, and predicting that tail directly should give a better safety–speed trade-off than simpler options. The simpler options are an average cost with a speed cap, a static risk map, or an average plus a multiple of the spread. If the conditional distributions turn out to be roughly bell-shaped, the last of these suffices, and the proposal is designed to reveal that early.

Core Claim

Conditional Value-at-Risk (CVaR) is the average cost over the worst α-fraction of outcomes. Let $Z(x, v, \theta)$ be the random per-step cost of crossing location $x$ at speed $v$ and heading $\theta$. The cost is measured from the vehicle's own sensors as vertical-acceleration bandpower and suspension travel extremes. We learn a network $f_\phi(x, v, \theta, \alpha) \approx \mathrm{CVaR}_\alpha(Z(x,v,\theta))$ that is conditioned jointly on speed and on the risk level α. We score a candidate trajectory with states $s_1, \dots, s_H$ by

$$ J_\alpha(s_{1:H}) = \sum_{t=1}^{H} f_\phi(x_t, v_t, \theta_t, \alpha) + J_{\text{task}}(s_{1:H}), $$

where $J_{\text{task}}$ is the usual progress and control-effort cost.

CVaR is sub-additive: the CVaR of a sum is at most the sum of the CVaRs. So this per-step sum is a conservative upper bound on the trajectory's true tail cost. It costs one network query per imagined state rather than a bundle of disturbance rollouts per candidate. How loose the bound is becomes an empirical question we measure directly rather than assume away.

The claim rests on this combination. Among the papers retrieved here, none learns a distribution that is both speed-conditioned and risk-conditioned, and none queries such a distribution inside a sampling-based controller over real 3D terrain.

Relation to Prior Work

Three works supply the pieces separately.

How Does It Feel? learns costmaps from camera and height data. It feeds speed in through Fourier features so the speed signal is not drowned out by the image, and it uses vertical-acceleration bandpower as a self-generated label. It is trained with mean-squared error, so it predicts only the average. We reuse its labels and its speed encoding.

Learning Risk-aware Costmaps trains a U-Net conditioned on α. It predicts Value-at-Risk (the α-quantile) with a Huber quantile loss, and CVaR with an L1 residual loss on the tail. Its costs are explicitly independent of speed and heading. We reuse its loss.

Risk-Conditioned Distributional SAC embeds the risk parameter with cosine features and resamples it within each mini-batch during training. It reports that training across the whole spectrum of risk levels improves robustness. We borrow that embedding and training scheme.

The planning side supplies the competitors and the motivation.

Risk-Aware MPPI computes empirical CVaR over nested disturbance rollouts. MPPI (Model Predictive Path Integral control) is a GPU controller that samples control sequences and weights them by cost. That paper shows tail-risk scoring cuts collisions sharply relative to average-cost MPPI, but only on flat 2D tracks with synthetic noise. It also shows that sample cost grows as α shrinks.

MPC for Aggressive Driving Over Uneven Terrain evaluates physics-based rollover and ditch inequalities on demand inside MPPI at 7–10 m/s. It is deterministic and exposes no risk dial. We add its constraints unchanged as a hard safety layer, so any gain we report comes from the learned tail term.

EVORA is the closest in spirit. It predicts traction distributions and forward-simulates their worst-case tail. It is planar, fixes the risk level during runs, and models traction rather than speed-dependent impact.

STEP motivates the whole effort. It exposes α only in a static long-horizon grid search and leaves speed to a separate tracking controller. That separation is exactly what lets a path be safe in geometry yet unsafe at the speed required to drive it.

Research Program

Labels and data. We build the training set from TartanDrive and TartanDrive 2.0. Together they give roughly 12 hours of all-terrain-vehicle driving at speeds up to 15 m/s, with heightmaps, RGB, IMU, wheel encoders and shock travel. For each short window of driving we pair three things: the terrain patch under the vehicle's footprint, the vehicle's speed and heading at that moment, and the realized cost. The cost combines z-axis acceleration bandpower and peak shock travel.

Network. A convolutional encoder runs once per frame over the local height-and-color map. A small multilayer perceptron head then takes three inputs: the feature sampled at a query location, a Fourier-feature encoding of speed and heading, and a cosine embedding of α. The encoder-then-head split is what makes queries cheap enough to run on demand; only the head runs per imagined state.

Training. We draw α uniformly per sample, following the resampling scheme of RC-DSAC. We use the quantile-plus-residual loss of the risk-aware costmap paper so the network outputs Value-at-Risk and CVaR together.

Unfamiliar terrain. For patches the encoder has not seen, we follow EVORA's lesson and add a normalizing-flow density score on the encoder features. When density is low, the query returns the maximum training cost. This is a deliberately crude fallback; it keeps the conservative behavior for unfamiliar ground separate from the learned tail and stops the two from being confused.

Controller. The network drops into an MPPI controller alongside the rollover and ditch inequalities. The engineering target is 10,000 rollouts over a horizon of about 40 steps at no less than 20 Hz on a single consumer GPU. That is roughly 400,000 head evaluations per cycle.

Baselines

The naive reference is average-cost MPPI using the speed-conditioned mean costmap of How Does It Feel?, with a speed-cap multiplier swept to trace its trade-off curve. It uses the same encoder, the same data, and the same rollout budget. It is the "do nothing new" system. That paper reports up to 57% fewer human interventions than occupancy-based navigation over traversals up to 3,150 m, so it is not a weak baseline.

The strongest existing tail-risk method is Risk-Aware MPPI. Its reported figures are 80% fewer collisions than baseline MPPI in simulation at nearly identical lap times, and 55.7% fewer on a 1/28-scale car, running at 27–81 Hz on an RTX 3090. We run it by drawing nested disturbance samples from a heteroscedastic average-cost model trained on the same data. We give it the same total GPU time per control cycle, so it trades candidates for nested samples.

The static tail-risk baseline is Learning Risk-aware Costmaps, retrained on TartanDrive and ignoring speed. Its reported cost is 48.94 ms per frame against 241.92 ms for handcrafted geometric analysis.

The physics-only baseline is the constraint-based MPPI of MPC for Aggressive Driving. It is reported up to 22% faster than a geometry-based planner.

All learned baselines receive identical data splits and the same hyperparameter search budget.

Two ablations isolate the key ingredients. The first removes speed conditioning from our network. The second replaces the direct CVaR head with a mean-plus-k-standard-deviations head, with k mapped from α under a Gaussian assumption.

"Beating" the baselines means the following. Our system's curve of time-to-goal versus tail events must lie strictly below every baseline's curve across the α sweep, with non-overlapping bootstrap intervals in at least the α ≤ 0.1 regime. It must also hold the 20 Hz real-time target.

Why not just…?

Why not just use the average-cost speed-aware costmap and cap the speed when terrain looks rough? A speed cap slows the vehicle everywhere a patch looks risky. It cannot choose a different line that is safe at speed, and Risk Conditioned Neural Motion Planning reports that risk-conditioned planning changes path geometry rather than merely rescaling speed. Average-cost scoring also dilutes rare impacts, which is exactly the failure Risk-Aware MPPI documents. Our route-divergence and trade-off-curve measurements test this directly. If the capped mean model's curve matches ours, the tail modeling is unnecessary and we will say so.

Why not just predict mean plus k standard deviations? This is the real threat. If cost given patch and speed is roughly Gaussian, a mean-plus-spread model matches CVaR at lower complexity. Our first experiment therefore measures tail shape on TartanDrive before any planning work. We will report the excess kurtosis of the cost distribution by speed bin, and the gap between true empirical CVaR and its Gaussian estimate. If that gap is small at high speed, we adopt the Gaussian head and reframe the contribution as speed-conditioned risk dialing. We will not argue for quantiles in that case.

Why not just run nested-sampling Risk-Aware MPPI on terrain? It needs a disturbance model of the terrain. It also needs many samples per candidate to resolve small α, and the file's own account of that method flags this cost growing as α falls. We compare at equal GPU time. If nested sampling is both calibrated and fast enough, the only remaining advantage of our approach is throughput, and the results will show whether that matters for success.

Why not just use the rollover and ditch physics constraints? They are deterministic and have no risk dial. They also do not model surface-driven impact, such as bumps under vegetation or loose rock at speed. We keep them in every system, so any gain reported is added on top of physics rather than replacing it.

Proposed Experiments

Offline checks. The first offline check is calibration. On held-out TartanDrive 2.0 trajectories, split by location rather than by time to avoid leakage, we measure how often the realized cost exceeds the predicted Value-at-Risk, compared with the nominal rate α, in speed bins up to 15 m/s. We also measure CVaR prediction error. No reported result on Lacuna exists for speed-binned tail calibration on TartanDrive, so this establishes the reference. The second offline check is the tail-shape diagnostic that decides the Gaussian question above. The third is the looseness of the sub-additive bound: we compare the summed per-step CVaR of logged trajectories against the empirical CVaR of trajectory-level cost across similar trajectories.

Closed-loop tests. We drive a field ATV-class or scale off-road platform over repeated courses with mixed bumps, off-camber turns and vegetation. Each system is swept over its dial: α for ours, the speed cap and k for the baselines. We record time-to-goal, the rate of vertical-acceleration exceedances above a damage threshold, violations of the rollover and pitch constraints, human interventions, and controller frequency.

Decisive checks. Three results would decide the question. First, our trade-off curve must dominate the capped-mean and static-CVaR baselines at small α. Second, turning α must change the chosen routes, measured by distance to the capped-mean baseline's path at matched average speed, and not just the speed profile. Third, the speed-ablated network must lose most of the gain, which would show that speed conditioning is the active ingredient. If the Gaussian-head ablation matches the full model, the tail-shape claim fails and the paper reports that negative result.