Spending Motor Headroom on Uncertain Obstacles: Safe Fast Flight Through Sparse-View 3D Maps

A small drone that builds its own 3D map from a few camera frames will see things that are not there. When a scene reconstruction is fit to only a handful of views, the optimizer can match those images by placing "floaters", semi-transparent blobs of fake geometry hanging in empty air, often right along the camera's line of sight. A safety filter that treats every piece of the map as a wall will brake or swerve hard to avoid these phantoms. On a fast quadrotor, a hard swerve pushes the motors to their thrust limits. Once a motor is pinned at its limit, the controller loses the ability to steer precisely, and the vehicle can crash while dodging something that never existed. The usual fixes are to slow down, which wastes the vehicle's agility, or to delete low-confidence geometry, which risks deleting a real wall.

We propose a different use of uncertainty. Obstacles the map is unsure about should not trigger evasion right away. Instead they should set aside motor headroom: the drone keeps its current course and reserves enough spare thrust to dodge the object later, if the object turns out to be real. Meanwhile the drone keeps approaching, which gives it new viewpoints. Those views tend to dissolve floaters, because fake geometry is inconsistent from angles it was not fit to. A real obstacle gets confirmed before the drone passes the last point from which it could still evade using its available thrust. We then handle it with an ordinary hard safety constraint. The actuator limits decide where that last point is. Reconstruction uncertainty decides how much headroom to hold back. The labels that teach the drone to tell floaters from real geometry come from real dense captures subsampled to sparse views, so this part does not depend on synthetic rendering.

Core Claim

The new object is a safety filter whose constraints depend jointly on per-obstacle existence confidence and on the vehicle's saturated dynamics. Here a safety filter is a small optimization run at every control step that minimally edits the planner's command so the vehicle never enters an unsafe set. The closest existing system is SAFER-Splat. It writes a control barrier function directly over the ellipsoids of a 3D Gaussian Splatting map built online at 15 Hz. A control barrier function is a scalar "distance to danger" whose rate of decrease the filter bounds. 3D Gaussian Splatting represents a scene as many small colored, semi-transparent ellipsoids. SAFER-Splat treats every ellipsoid as a real obstacle, models the drone as a double integrator with unlimited acceleration, and relies on dense RGB-D streams. Even then it reports brief 1–4 mm safety violations, which the authors attribute to low-level tracking error. We change all three assumptions: the map comes from sparse RGB views, each obstacle carries an existence probability, and the barrier is measured against a saturation-aware point of no return rather than against the obstacle surface.

For a cluster of Gaussians $i$ with center $c_i$, radius $r_i$, and estimated existence probability $q_i$, and a vehicle at position $p$ with velocity $v$, we define

$$h_i(p, v) = \lVert p - c_i \rVert - r_i - D(v, \bar{T}),$$

where $D(v, \bar{T})$ is the minimum distance the vehicle needs to swerve clear when collective thrust is capped at $\bar{T}$. We learn $D$ offline from saturated rollouts, in the same spirit as the tracking-error model in Quad-LCD. Clusters with high $q_i$ get the standard barrier condition $\dot h_i \ge -\alpha h_i$, where $\alpha$ is a positive gain, so the vehicle turns away early.

Low-confidence clusters get only the commit constraint $h_i \ge 0$, together with a headroom reservation. The nominal command may use at most a fraction $1 - \rho$ of available thrust, where

$$\rho = \min\Big(\rho_{\max},\ \kappa \sum_{i:\, h_i < H} q_i\Big),$$

$H$ is a lookahead distance, and $\kappa$ is a gain. In words, the drone may fly straight at an uncertain object at speed, but only while it keeps enough reserved thrust and distance to dodge it on confirmation.

The claim is empirical. On a curve plotting time-to-goal against collision rate, this filter should dominate both the treat-everything-as-real filter and the obvious slow-down or prune-it fixes, in scenes reconstructed from 3–10 views.

Research Program

The work has three coupled pieces.

The first piece is an existence-confidence estimator for Gaussian clusters. It combines two signals. One is the matching confidence that feed-forward reconstructors already expose: MVSplat predicts Gaussian opacity from cross-view cost-volume matching and runs at about 22 FPS. The other is temporal consistency across newly acquired frames, measured as the disagreement between a cluster's rendered depth and freshly predicted depth from a new pose. This operationalizes RegNeRF's observation that floaters appear as spiky depth discontinuities from unobserved viewpoints. RegNeRF uses that fact to regularize an offline optimization. We use it online as evidence for classifying obstacles. A small classifier maps these features to $q_i$.

Training labels come from real captures. We densely photograph a space, reconstruct it with hundreds of images as Quach et al. did with 430 iPhone images, and treat that dense model as geometric ground truth. We then subsample the captures to 3–10 views, reconstruct again, and label each sparse-view cluster as real or floater by whether it is supported by the dense model. DTU provides object-scale scenes with reference geometry for validating these labels. Deriving labels from real imagery addresses the sim-to-real concern on the perception side. A classifier trained on synthetic-renderer floaters could instead learn renderer-specific artifact statistics, which is the shortcut failure described in the sim-to-real literature.

The second piece is the saturation model $D(v, \bar{T})$ and the filter itself. Following Quad-LCD, we generate a large number of RotorPy rollouts with parasitic drag and motor noise, including cases where individual motors hit their limits under a geometric controller. We fit $D$ as the minimum lateral-evasion distance over initial speeds and headings. The filter is a quadratic program over the thrust and moment command, with the standard barrier rows for confident clusters, the commit rows for uncertain ones, and the headroom cap. Under the cap, moments are allocated by priority: yaw is sacrificed first and roll/pitch are preserved. This follows the logic-based prioritization that Song et al. use for wheel torque distribution when tires reach adhesion limits, and it avoids the directional distortion that clipping each motor independently causes.

The third piece is the closed-loop testbed, which combines a sparse-view 3DGS digital twin with separate, saturation-faithful dynamics. Flightmare shows that decoupling a fast physics loop from a slower renderer is practical. We couple RotorPy dynamics to 3DGS rendering of the same real scenes. The sparse-view reconstruction is rebuilt online from frames the simulated drone actually captures, so floaters arise naturally from the flight's own viewpoints rather than being injected. The dense reconstruction supplies collision ground truth. Because the dense and sparse models come from the same real capture, "the drone hit something" means it hit real geometry. This lets us score false evasions and true collisions separately, which no benchmark in our file currently does.

Baselines

The naive reference point is SAFER-Splat's design applied unchanged to the sparse-view map. Every Gaussian is an obstacle, the filter uses double-integrator dynamics, and a geometric controller clips motors independently. This is the direct competitor on the question "does taking the map at face value fail under sparse views?" It will run on identical scenes, start/goal pairs, reconstruction pipeline, and compute. SAFER-Splat reports 1–4 mm violations and a 15 Hz loop, but no reported result found on Lacuna for its behavior on sparse-view maps. Beating it means fewer false evasions and saturation-induced crashes at equal or lower true-collision rate.

The strongest existing method for the dynamics side is Quad-LCD's controller-aware tracking penalty, composed with the same all-Gaussians barrier filter. In aggressive RotorPy tests Quad-LCD reports a 6% crash rate versus 41% for minimum-snap geometric control (MS-GC) and 54% for MS-GCD, as reported in Quad-LCD. That result is in free space with known waypoints, which makes this composition the fairest "existing parts glued together" competitor. We will retrain its penalty model on our RotorPy configuration with the same rollout budget we use for $D$. Beating it means a better time-to-goal versus collision curve once phantom obstacles are present.

Two uncertainty-handling baselines complete the set. The speed-capped filter limits velocity so that no evasion can saturate the motors. The pruned-map filter drops every cluster below a tuned confidence threshold, using our own estimator so the comparison isolates the decision rule. Both are tuned by grid search on held-out scenes, with the same number of configurations as our $\kappa$, $\rho_{\max}$, and $H$ get.

Two ablations isolate the key ingredient. Setting $q_i = 1$ for all clusters keeps saturation awareness but removes perception coupling. Replacing $D(v, \bar{T})$ with a fixed safety margin keeps confidence weighting but removes actuator coupling. If either ablation matches the full method, the joint dependence we claim is not doing the work.

Why not just…?

Why not just fly slower? This is the most serious threat. At low enough speed, any evasion is actuator-feasible and phantom obstacles cost only small detours. It will beat the naive baseline on crashes. The question is what it costs in time. Quad-LCD's results show that aggressive trajectories are where saturation-induced crashes concentrate, so the regime that matters is precisely the fast one. We settle this by plotting the full time-to-goal versus collision-rate curve for the speed cap across its tuning range. If the speed-capped curve reaches ours, the idea has failed and we will say so.

Why not just prune low-confidence Gaussians before planning? Few-Shot Novel View Synthesis Using Depth Aware 3D Gaussian Splatting found that with 5 views, standard opacity pruning removes essential real geometry, and it had to keep every splat. Floaters under the shape-radiance ambiguity described by RegNeRF can look as opaque as real surfaces from the training views. Pruning is therefore an irreversible bet made at the moment of least information. Our filter defers that bet until more views arrive. The pruned-map baseline will show which way the trade falls. We expect it to either miss real thin obstacles or keep enough floaters to reproduce the naive failures, depending on the threshold.

Why not just use Quad-LCD's planner with a standard barrier filter? Quad-LCD makes the nominal trajectory feasible in free space. It cannot anticipate a filter override triggered by a phantom that appears mid-flight, and it has no notion of obstacle confidence. That composition is included as a baseline precisely because it might be enough. The $q_i = 1$ ablation of our method is close to it and tells us whether confidence weighting adds anything beyond saturation awareness.

Why not just skip the metric map and use a frozen foundation-model encoder, or a depth sensor? FAINT reaches 90% real-world success with a frozen encoder trained purely in simulation. However, it outputs actions without metric geometry, so there is nothing for a barrier function to act on. It was also evaluated at speeds where saturation does not arise. WMP shows learned latents handling tight clearances, but with a depth sensor on a legged robot. A good active depth sensor would indeed remove most floaters. Our claim is scoped to RGB-only, weight-limited platforms, and we state that scope rather than argue around it.

Proposed Experiments

The first experiment validates the confidence estimator. On DTU scenes and on our own dense-then-subsampled flight-space captures, we measure how well $q_i$ separates floaters from real clusters (area under the ROC curve) at 3, 5, and 10 views. We track how quickly the separation improves as online frames accumulate along straight approach paths. Reconstruction quality is sanity-checked against published figures. RegNeRF reports PSNR 18.89 on DTU with 3 views, and MVSplat reports PSNR 26.39 on RealEstate10K. Our estimator's value, however, is judged by classification and by closed-loop outcomes, not by image quality.

The main experiment is closed-loop aggressive flight in the sparse-view digital twin. We use at least six real indoor scenes with thin and overhead obstacles, held-out start/goal pairs, and target speeds swept up to the platform's limits. Metrics are true-collision rate against the dense model, false-evasion rate (filter overrides caused by clusters absent from the dense model), fraction of time with any motor saturated, and time-to-goal.

The decisive check has two parts. At matched true-collision rate, our filter must reduce time-to-goal relative to the tuned speed cap and reduce saturation time relative to the Quad-LCD composition. Both ablations must fall measurably short of the full method. A diagnostic accompanies this: the fraction of real obstacles confirmed ($q_i$ above threshold) before the vehicle crosses $h_i = 0$. If that fraction is not near one at the speeds where we claim gains, the method is winning by luck rather than by resolving uncertainty in time.

A final hardware stage flies a camera-equipped quadrotor in one captured room with physical obstacles and deliberately sparse initial views. This stage reports the same metrics over repeated trials, to test whether the saturation model $D$ learned in RotorPy and the real-data-trained confidence estimator transfer together.