Motion Blur as a Sign-Free Velocity Sensor for Visual-Inertial Robots

A camera on a fast robot does not take instantaneous snapshots. Each frame adds up light over the whole time the shutter is open, so a fast-moving drone records streaks instead of sharp edges. Robotics treats those streaks as damage: features vanish, visual odometry (tracking the robot's motion from its camera) loses lock, and the robot either falls back on drifting inertial dead-reckoning or needs an external motion-capture room to tell it where it is. Vision research has shown the opposite. A blurred image is a compressed record of motion during the exposure, and the length and orientation of the streaks can be decoded into velocity. That decoded motion has one flaw. Because summing light over time gives the same answer whichever way time runs, a streak cannot tell you which way the camera moved.

This proposal treats that flaw as a feature of the measurement and builds the estimator around it. Each blurred stereo frame is converted into a sign-free measurement of the robot's velocity during the exposure: it gives the axis and the speed, but it is symmetric under reversing the direction. The inertial measurement unit (IMU, the accelerometer and gyroscope every drone carries) is then used to supply the missing sign. The physics of which IMU channel can supply it, and when that supply fails, is the core of the proposal. The result would be an onboard velocity estimator that gets better information from standard frame cameras exactly when motion is fast, with no event camera and no motion capture.

Core Claim

The claim has two parts. The first is that blur contains usable information about translational velocity that current visual-inertial pipelines throw away. Streak extent scales with $|v_\perp|\tau/Z$, where $v_\perp$ is the velocity component perpendicular to the viewing ray, $\tau$ is the exposure time, and $Z$ is scene depth. Stereo depth survives blur because both cameras smear identically under a synchronized exposure. Fusing this measurement with the IMU should reduce velocity drift in the blurred, low-light regimes where Blackbird, TUM VI, and TUM-VIE show frame-based estimators degrading.

The second part, which is less obvious, concerns where the sign of motion actually comes from. Under time reversal, angular velocity changes sign, so the gyroscope resolves the direction of rotational blur immediately. Acceleration does not change sign under time reversal, because it is a second derivative. The accelerometer therefore cannot break the ambiguity within a single frame, even though the uneven brightness along a streak from speeding up or slowing down looks like a cue. The translational sign can only come from velocity, which the IMU knows only by integrating acceleration, and that integral drifts. Writing $\hat v$ for the IMU-propagated velocity estimate along the blur axis and $\sigma_v$ for its standard deviation, a Gaussian prior assigns the wrong mode with probability

$$ P(\text{sign error}) \approx \Phi\!\left(-\frac{|\hat v|}{\sigma_v}\right), $$

where $\Phi$ is the standard normal cumulative distribution. This is negligible in steady tracking. It becomes large after vision dropouts, at start-up, or near velocity reversals, which are the moments blur-aware estimation is meant to help with. We predict that an estimator which hard-assigns the sign from the IMU prior will fail in these regimes. An estimator that carries both hypotheses should recover. If that prediction is wrong, the experiments below will show it cleanly.

Relation to Prior Work

The direction ambiguity has been characterized and then avoided. Optical Flow Estimation from a Single Motion-blurred Image decodes a blurred frame into flow and trains with a loss that takes the minimum over forward and backward flow. Its output is therefore sign-free by construction, and the authors note that an IMU or a neighboring frame would be needed to fix the direction. Parametric Object Motion from Blur sidesteps the issue by anchoring the sharp image at mid-exposure. Other work motivates the gap by showing that unconstrained unfolding produces reversed or implausible sequences: Purohit et al. and Rozumnyi et al.. We supply the method: we reuse the Argaw-style sign-free blur decoder as the visual front end, and we resolve the sign downstream in the state estimator rather than in the network.

Two other families resolve direction with extra signals. Event cameras provide it in hardware, as in BeNeRF and the Event-based Visual Inertial Velometer. The velometer is the closest robotic analogue: it estimates map-free instantaneous velocity from stereo events plus IMU on continuous-time splines. We keep its map-free, velocity-first design, but replace events with blurred frames and must therefore handle a sign the events gave for free. Generative models guess the direction instead: Generating the Past, Present and Future from a Motion-Blurred Image and Diffusion-based Event Generation produce plausible motion without metric or physical guarantees.

On the robotics side, MBA-VO is the strongest blur-aware odometry. It re-blurs sharp keyframes along a start-to-end pose interpolation and aligns them photometrically. It is monocular, has no IMU, relies on frame order for coarse direction, and needs sharp keyframes. BAD-Gaussians uses cubic B-spline trajectories inside the exposure, but optimizes offline for about 30 minutes. Ctrl-VIO supplies our continuous-time machinery: cumulative $SE(3)$ B-splines fused with high-rate IMU. It models rolling-shutter row timing but treats each row as instantaneous, so it never uses blur. Sun et al. show frame-based VIO collapsing during rotor-failure spins and switch to events. This constrains our ambitions: at spin rates above 20 rad/s, rotation dominates the streaks, and we treat that regime as a stress test we may lose.

Research Program

The system has three parts. The first is a blur decoder. It is a network in the style of Argaw et al., trained on GoPro and Monkaa and on blur synthesized from rendered sub-frames. It outputs a per-pixel unsigned motion field over the exposure, together with a confidence. The confidence matters because textureless regions produce degenerate streaks, a limitation both Gast and Argaw report. In those regions the confidence should drive the likelihood toward flat, so the system falls back to the IMU rather than trusting noise.

The second part is a sign-symmetric measurement model. Given stereo depth and the camera intrinsics, the unsigned field constrains the body twist $\xi = (\omega, v)$ during exposure through a likelihood that satisfies

$$ p(B \mid \omega, v) = p(B \mid -\omega, -v), $$

where $B$ is the blurred stereo pair. The equality is exact under uniform exposure, constant twist within the frame, and a static scene. Because gyroscope readings fix $\omega$ within noise, the residual symmetry reduces to the sign of $v$ along the blur axis.

The third part is a continuous-time estimator that carries both hypotheses. We use a Ctrl-VIO-style spline over the IMU stream. The blur term enters as a two-component mixture on velocity rather than being linearized around one mode. Hypotheses are pruned when either the propagated prior or a cross-frame cue becomes decisive. The cross-frame cue is continuity: the end of the streak in frame $k$ should lie near the start of the streak in frame $k+1$. The analysis half of the program measures three things: how often each regime occurs in real flight logs, which cue actually resolves the sign in each regime, and whether the $\Phi(-|\hat v|/\sigma_v)$ prediction matches observed sign-error rates.

Baselines

The naive reference is IMU-only propagation through blurred intervals: the visual front end is simply dropped whenever features are lost. It sets the floor that any blur measurement must beat. It shares our IMU noise model and uses no extra tuning.

The strongest frame-based visual-inertial competitors are Basalt and VINS-Fusion, run as released on TUM VI and TUM-VIE with their default configurations plus an equal-effort tuning pass on exposure-related thresholds. On TUM-VIE bike-dark, TUM-VIE reports ATE of 6.26 m for Basalt and 20.20 m for VINS-Fusion. Beating Basalt's 6.26 m on that low-light, long-exposure sequence is the headline target for full-trajectory accuracy.

The strongest blur-aware method is MBA-VO, which reports 0.1021 m ATE on TUM-RGBD fr1-desk against 0.4956 m for DSO, at 34.4 ms per frame on an RTX 2080. We run it monocularly as published. We also build a variant with an IMU prior, since the authors name visual-inertial extension as future work, so that our advantage cannot be credited merely to having an IMU. Both are compared on the same hardware.

The event-camera reference is the Event-based Visual Inertial Velometer. It reports an average velocity error of 0.06 m/s in simulation and 0.72 m/s on a real rig spinning above 13 rad/s, at about 75 Hz. We treat it as an upper bound that uses extra hardware, and we run it on TUM-VIE, which records events and frames on the same rig. Matching it within a small margin using only frames would be a strong result, and beating it is not required.

Four ablations isolate the key ingredient. The first removes the blur term. The second resolves the sign by hard assignment from the IMU prior instead of carrying a mixture. The third resolves it by frame chaining only. The fourth uses the gyroscope only, with translational sign left to the prior.

Why not just…?

Why not just shorten the exposure? In dim scenes, shorter exposures trade blur for noise and higher gain. Both destroy the image gradients odometry depends on, which is the regime MBA-VO targets and the regime in which TUM-VIE bike-dark defeats frame VIO. We test this directly on TUM VI sequences and simulated flights by sweeping exposure against a fixed photon budget. If a short-exposure Basalt matches our method at every light level, the proposal's practical case collapses, and we would report that.

Why not just mount an event camera? Events do solve direction in hardware, but they are absent from most commodity platforms, as the diffusion-event paper stresses. TUM-VIE's authors also call event odometry immature. The event velometer baseline measures exactly how much accuracy one gives up by staying with frames.

Why not deblur each frame and then run standard VIO? MBA-VO reports that per-frame deblurring is slow and introduces artifacts that are inconsistent across frames and break photometric and feature tracking. Argaw et al. report that decoding motion directly from blur (2.038 EPE on GoPro) beats deblurring first and then computing flow. We include a deblur-then-Basalt pipeline to confirm this on robot data.

Why not just take the sign from the IMU and skip the mixture? This is the real threat. In steady flight, $\sigma_v \ll |\hat v|$, and hard assignment will be right almost always. Our claim rests entirely on dropout, start-up, and reversal regimes. The hard-assignment ablation, evaluated separately on segments stratified by $|\hat v|/\sigma_v$, settles the question. If it matches the mixture everywhere, we will conclude the ambiguity is practically moot once an IMU is present. The contribution then narrows to blur as a velocity measurement, which remains testable on its own.

Why not just chain consecutive frames? Streak continuity across frames resolves the sign without the IMU and may suffice at high frame rates. Blackbird's 120 Hz stereo favors it, while TUM VI's 20 Hz frames leave larger gaps. The chaining-only ablation measures this, and we expect chaining and the IMU prior to be complementary rather than one dominating.

Proposed Experiments

We train the decoder on GoPro and Monkaa and first confirm that it matches Argaw et al.'s reported 1.158 and 2.038 end-point error on the unsigned flow. For robot data we use TUM VI (stereo 20 Hz, 200 Hz IMU, motion-capture segments) and TUM-VIE (frames, events, and IMU on one rig, including bike-dark). We also use Blackbird, re-rendering blurred frames by averaging FlightGoggles sub-frames if the rendering pipeline permits. Blackbird is the speed-controlled test: its trajectories fly the same path at different speeds, and its reported drift rises from 0.55% to 1.94% as speed goes from 1.12 to 3.08 m/s. The metrics are average velocity error and ATE against motion capture, plus sign-error rate per frame and time to recover a correct velocity after an injected vision dropout.

Three checks are decisive. The first is whether, on Blackbird, our drift grows with speed more slowly than the baselines', since blur information should grow with speed. The second is whether, on segments with low $|\hat v|/\sigma_v$ created by injected dropouts and start-ups, the observed sign-error rate of the hard-assignment ablation tracks the $\Phi$ prediction and the mixture removes those errors. The third is whether, on TUM-VIE bike-dark, we beat Basalt's 6.26 m ATE using frames and IMU only. A rotor-failure-style spin simulation in ESIM is reported as a stress test with no success promised.