Heading Agreement: Radar Doppler as a Live Check on Drifting Sensor Calibration

Camera–LiDAR–radar fusion depends on knowing exactly how each sensor is mounted relative to the others. That relative pose is the extrinsic calibration. Fusion pipelines treat it as a constant measured once in the garage. On real vehicles the mounts move. Cabs pitch on their suspension, brackets vibrate, and structures flex with load and temperature. A fraction of a degree of camera rotation shifts distant LiDAR points by several pixels, and that is enough to paint pedestrian labels onto the background.

This note proposes a cheap, appearance-free way to watch the calibration while driving. A Doppler radar measures, in a single scan and without matching anything between frames, the direction its own mount is travelling. A camera reveals the same direction as its focus of expansion, the image point that the scene appears to stream away from. If the two sensors were rigidly locked together, these two headings would agree once mapped through the nominal calibration. Any disagreement is a direct, frame-by-frame measurement of how far the mount has rotated.

We propose to estimate time-varying rotational misalignment from this heading disagreement. We would model the misalignment the way deformable surgical reconstruction models tissue: a fixed canonical pose plus a smooth, low-dimensional offset that changes over time. The corrected, uncertainty-tagged projection would then feed a cross-attention fusion network. The claim is narrow and testable. Single-scan Doppler heading recovers 2 of the 3 rotational degrees of freedom of mount drift at driving speeds, with accuracy that improves as the vehicle goes faster. Feeding that estimate to fusion closes most of the accuracy gap between nominal and oracle calibration, at a cost far below dense recalibration.

Core Claim

The physics is the single-scan Doppler equation. Point $i$ returns a radial velocity $v_{r,i} = \mathbf{v}\cdot\mathbf{u}_i$, where $\mathbf{v}$ is the sensor's 3D velocity in its own frame and $\mathbf{u}_i$ is the unit direction to the point. A robust least-squares fit over the stationary points of one scan therefore gives $\mathbf{v}$ directly. Doppler-only Single-scan 3D Vehicle Odometry does exactly this, with RANSAC to reject moving objects, in about 4.5 ms per scan. That work, and Kubelka et al., use the result for odometry.

We use only its direction $\hat{\mathbf{v}}$, and we use it as a calibration observable. Write $\mathbf{R}_0$ for the nominal radar-to-camera rotation and $\mathbf{R}_{\delta(t)}$ for a small unknown drift with parameters $\delta(t)$. Write $\hat{\mathbf{f}}(t)$ for the camera's heading ray, computed from the focus of expansion after removing rotational flow. We estimate the drift over a sliding window of times $t_k$ as

$$ \delta^\star = \arg\min_{\delta}\; \sum_k \rho\!\left(\angle\big(\mathbf{R}_{\delta(t_k)}\mathbf{R}_0\,\hat{\mathbf{v}}(t_k),\ \hat{\mathbf{f}}(t_k)\big)\right) + \lambda\,\|\partial_t^2 \delta\|^2 , $$

where $\rho$ is a robust loss, $\angle$ is the angle between two unit vectors, and $\lambda$ weights a smoothness prior on the drift trajectory.

Two pieces of structure make this tractable. First, one heading pair constrains only the two rotation axes perpendicular to the direction of travel. Roll about the travel axis becomes observable only during turns, or from the ground plane, so the estimator reports per-axis uncertainty rather than pretending to full observability. Second, the angular resolution scales roughly as $\sigma_v/\lvert\mathbf{v}\rvert$, where $\sigma_v$ is the noise of the Doppler velocity fit. This follows from the error of the least-squares direction estimate. The method should therefore be accurate at highway speed and should honestly abstain near standstill.

The same construction handles radar–LiDAR drift. The LiDAR reference heading comes from scan-to-scan ego-velocity. Rotating a radar cloud rotates its Doppler directions $\mathbf{u}_i$ consistently, so perturbations can be injected exactly.

How the Four Directions Fit

Radar-only ego-motion supplies the measurement. Its key property is that a Doppler heading needs no data association and does not depend on appearance. DoGFlow showed the practical value of this on MAN TruckScenes: Doppler-guided scene flow cut snowy-condition error from about 1.70 m to 0.90 m relative to geometry-based baselines.

Calibration sensitivity motivates the work. Peters and Brenner show that a 5-pixel shift is harmless for trucks but makes labels bleed off thin structures. Tan et al. and Yang et al. document misalignment that compounds under motion and varies over time. Liu et al. identify mechanical oscillation as a driver of localization failure. DoGFlow itself is acknowledged to be sensitive to radar-to-LiDAR extrinsics but does not estimate them. That is a concrete consumer for our estimate.

Feature fusion is the application. SGFormer lets 3D points query 2D feature maps through cross-attention instead of fetching a single projected pixel. It gains 2–10% under sensor stress tests, but it still assumes a fixed calibration. We keep its querying mechanism and change two inputs: the projection centre comes from the drift estimate, and the attention window scales with the drift uncertainty.

The non-rigid tissue literature is the modeling lens, not the application. Forplane and Endo-4DGS both represent a deforming body as a canonical state plus a compact time-dependent offset field. We treat the sensor rig the same way. A slow component captures settling and thermal change, and an oscillatory component captures suspension pitch. Both sit on top of the factory calibration.

The surgical work also supplies a cautionary result we take seriously. EndoFlow-SLAM reaches 0.23 mm trajectory error (ATE) on the C3VD colonoscopy dataset but degrades to 15.47 mm on deforming StereoMIS tissue. Its 2D optical flow mixes ego-motion with scene deformation. Our design separates the two by construction: the heading reference comes from Doppler on stationary returns, so it does not depend on image flow at all. Extending this to surgery would require Doppler-capable endoscopic probes that no dataset in the file provides, so we do not claim it.

Research Program

The first stage asks whether the problem exists on real data. On MAN TruckScenes we would compute, frame by frame, an estimator-independent alignment score: the agreement between LiDAR depth discontinuities and image edges under the factory calibration. We would then test whether this score varies with braking, road roughness, or speed. If it is flat, real drift is negligible on this platform, and the contribution reduces to robustness under injected drift. We would report that outcome plainly.

The second stage builds the estimator. It has four parts:

  1. Single-scan Doppler heading with RANSAC on stationary returns, following Galeote-Luque et al..
  2. A focus-of-expansion estimate from sparse flow, with rotational flow removed using the radar-plus-kinematics angular rate or a gyroscope.
  3. The windowed robust fit above, parameterized as a canonical offset plus a low-order temporal basis.
  4. Per-axis covariance computed from the fit.

The third stage connects the estimate to two consumers. One is an SGFormer-style panoptic segmentation network on nuScenes, which receives the corrected projection and uncertainty-scaled attention windows. The other is DoGFlow's pseudo-label propagation on TruckScenes, which receives the corrected radar–LiDAR extrinsics.

Controlled drift is injected without simulation artifacts:

  • Camera rotation drift is applied as an exact homography warp of the images. A rotation about the camera centre is exactly a homography, so the camera's own focus of expansion moves consistently with the injected drift.
  • Radar rotation drift is applied by rotating both the points and their line-of-sight vectors.

Drift profiles cover three regimes: slow ramps, oscillations at suspension-like frequencies, and step shocks. Amplitudes run from 0.1° to 2°.

Baselines

The naive reference is the fusion network run with the factory calibration under each injected drift profile. The ceiling is the same network given the oracle, drift-corrected calibration. Every other method is scored by the fraction of the gap between these two that it closes.

The strongest calibration competitor is UniCalib. It converts both camera and LiDAR to dense depth, matches them with probabilistic flow, and solves for the extrinsics with PnP and RANSAC. It reports 0.550 cm translation and 0.044° rotation error on KITTI at about 11 Hz. That accuracy is far finer than our drift amplitudes, so it is a serious threat. We would run it per frame and over sliding windows on the same injected sequences, at the same GPU budget, with its released settings. For radar–LiDAR drift, where UniCalib does not apply directly, we would substitute LiDAR-referenced heading as the competitor. Our method beats UniCalib if it matches UniCalib's time-resolved rotation error within a factor of two, at under a tenth of the compute. It beats it outright if it is more accurate in snow or low-texture segments, where depth estimation is weakest.

The strongest fusion competitor is SGFormer trained with the same budget. It reports 80.9% panoptic quality (PQ) on the nuScenes validation set with clean calibration. No reported result was found on Lacuna for any method under time-varying extrinsic drift, so we establish that number ourselves.

A fair version of the fusion competitor must also see drift during training. We therefore train it with random extrinsic perturbation augmentation drawn from the same amplitude distribution as the test drift, using the same number of epochs and the same hyperparameter search.

Two ablations isolate the key ingredient:

  • Measurement ablation. Replace the Doppler heading with the LiDAR scan-matching heading, keeping the estimator identical. This tests whether Doppler specifically matters.
  • Uncertainty ablation. Feed the corrected calibration to plain projection instead of uncertainty-scaled attention. This tests whether the uncertainty is used.

Why not just…?

Why not just train the fusion network with calibration-noise augmentation? This is the main threat. Augmentation teaches the network to tolerate a spread of offsets by widening its effective receptive field, and that may cost accuracy on thin structures even when calibration is good. It cannot track an offset that changes over time. Our experiment settles this directly. We compare augmented SGFormer against corrected-input SGFormer on clean data, on slow drift, and on oscillatory drift, broken down by object size. If augmentation closes the gap equally well for pedestrians and poles, the estimator is unnecessary for fusion, and its value shrinks to its other consumers such as DoGFlow pseudo-labels.

Why not just rerun a targetless calibrator like UniCalib online? UniCalib is very accurate on KITTI, but it solves for one rigid transform and depends on a monocular depth network, which is its own stated runtime bottleneck. Its outlier weighting treats moving regions as uninformative, and it was not designed to track oscillation. Running it in a sliding window measures exactly how well it tracks. The Doppler heading residual costs one small least-squares fit per scan. Our claim is about cost and appearance independence, not about beating UniCalib on clean static calibration.

Why not just use LiDAR scan matching, or an IMU, as the heading reference? LiDAR ego-velocity is a fine reference in structured, clear-weather scenes, and the measurement ablation may show it equals radar there. Radar's expected advantage lies where scan matching degenerates, as in tunnels and open highway, and in snow, where DoGFlow's Doppler guidance halved error. IMUs give good angular rate but drift in translation and heading, as Galeote-Luque et al. report. They also mount on yet another bracket whose calibration can drift. If the LiDAR variant matches radar in TruckScenes snow segments, we would conclude that the Doppler-specific claim fails.

Proposed Experiments and Decisive Checks

On nuScenes we would inject camera rotation drift and report three things: time-resolved rotation error in degrees RMS per axis against the injected truth, panoptic quality, and mIoU split by object size. The gap-closed fraction is the headline number.

On MAN TruckScenes we would inject radar–LiDAR drift and measure the change in DoGFlow pseudo-label end-point error when its extrinsics are corrected. We would also run the real-data alignment-score analysis from the first stage.

Four checks would decide the proposal:

  • Null test. With no injected drift on segments where the alignment score is stable, the estimator must output near-zero drift. A spurious nonzero estimate there would mean it is fitting noise.
  • Speed curve. Rotation error should fall roughly as $1/\lvert\mathbf{v}\rvert$, and the estimator should abstain at low speed. If the curve is flat, the error is dominated by the focus-of-expansion estimate rather than by Doppler noise.
  • Observability test. Roll error should remain large on straight segments and shrink during turns, confirming the reported uncertainty is honest.
  • Augmentation comparison on thin classes. If our method does not beat drift-augmented SGFormer on pedestrians and poles under oscillatory drift, the fusion claim is withdrawn.