Rigid-Part Swept Bounds for Planning Around Cooperatively Seen Agents

A car approaching an intersection cannot see the cyclist hidden behind a bus. A roadside sensor can, but its view arrives 100 to 200 milliseconds late over a wireless link. By then the cyclist has moved. A planner that checks collisions only at discrete instants can also miss a contact that happens between two checks. This failure is called collision tunneling: the planned path passes through an obstacle that no individual check catches. This proposal asks a narrow question. Once a moving agent has been reconstructed from shared sensors as a cloud of 3D Gaussians (small ellipsoidal blobs, the primitives of Gaussian splatting), can we compute a region of space-time that provably contains that agent over the whole latency-plus-planning window? The region must also be tight enough that the ego vehicle does not freeze.

The key idea is that rigidity is what makes such a bound computable. If every Gaussian has its own free velocity, as in current 4D decomposition methods, bounding where thousands of blobs might go yields either nothing useful or a huge envelope. If instead the agent is regularized to move as-rigidly-as-possible, the cloud collapses to a few near-rigid parts. "As-rigid-as-possible" (ARAP) means a penalty that makes each local patch move like a rotated, translated copy of itself. Each part's reachable motion over an interval has a closed-form envelope. That envelope inflates each attached Gaussian, and the ego trajectory is kept away from the inflated Gaussians by a hard convex constraint rather than a soft penalty. ARAP regularization therefore stops being a cosmetic prior for reconstruction. It becomes the structural assumption that turns a dynamic, delayed, cooperatively observed scene into a set of certified, tunneling-free obstacles.

Core Claim

We claim the following. A 4D Gaussian reconstruction of moving agents can be built from fused vehicle and infrastructure views, with ARAP regularization on a coarse deformation graph per agent. From it, one can derive per-part swept bounds that account for communication latency, kinodynamic limits, measured non-rigid residual, and cross-agent registration error. These bounds can be enforced as hard constraints on the convex hull of a spline trajectory. Compared with the strongest box-based alternative, they should miss ground-truth future occupancy no more often. They should also be measurably less conservative around large, articulated, or non-rigid agents such as trucks with trailers, buses, cyclists, and pedestrians. We do not claim an advantage for ordinary sedans, where a rigid box is already close to the true shape. That limit is a prediction we will test, not a caveat to hide.

How This Differs From the Closest Work

Four lines of work set up the problem, and each stops short of it.

FOCI computes a closed-form, differentiable overlap integral between a robot's Gaussians and a 3D Gaussian splat map, and wraps trajectories in tight convex hulls built from cubic B-spline MINVO bases. Its authors state it is restricted to static maps. Collision appears there as an additive penalty, so the trajectory can clip through thin surfaces when smoothness terms dominate. We reuse its MINVO hull machinery for the ego. We add two things it lacks: moving obstacles with bounded swept extent, and hard rather than soft separation.

CIAO supplies the other half of the method. It guarantees inter-sample safety with an "action radius" derived from velocity and acceleration limits, inside convex free balls. However, it approximates moving humans as discs. We lift its action-radius idea from discs to parts of a deformation graph.

Implicit Swept Volume SDF handles arbitrary deforming shapes over the full time interval through semi-infinite programming. It assumes the deformation is known, assumes obstacles are static, and is too slow for real-time 3D use. Our bound is coarser but closed-form, and it applies to observed, uncertain deformations.

On the reconstruction side, two papers motivate the gap. CRUISE builds decomposed Gaussian scenes from vehicle-infrastructure captures, treats vehicles as rigid box-tracked clusters, and generates trajectories without any continuous collision check. Split4D separates moving from static Gaussians natively in 3D using per-primitive linear velocities. Its Gaussians bleed across contact boundaries because nothing ties neighbours together.

The method for the rigidity term comes from ARAP-GS. It runs ARAP on a subsampled anchor graph of Gaussian centres and propagates motion to all Gaussians by nearest-neighbour interpolation. Its 10 to 20 minute offline solve is a constraint we must work around. ARAPReg and the elastic Jacobian penalty of Nerfies show that such priors separate real motion from artifacts.

The cooperative-tracking papers define the latency regime we must beat:

  • DAIR-V2X introduces Time Compensation Late Fusion (TCLF), which projects boxes forward under constant velocity.
  • V2X-Seq introduces FF-Tracking, which warps delayed infrastructure features forward with predicted flow.

Both produce point estimates of where an object will be, not envelopes of where it could be.

Research Program

Reconstruction. Each dynamic agent is built from fused ego and infrastructure frames. Its Gaussians are attached to a coarse deformation graph: a few dozen anchor nodes for a vehicle, more for a pedestrian, following the ARAP-GS anchoring scheme. Each node carries a rigid transform per frame, and an ARAP energy penalizes deviation of neighbouring nodes from a shared local rotation.

To fit the streaming setting, the graph is not re-solved from scratch each frame. Node transforms are warm-started from the previous frame and refined for a fixed small number of Gauss-Newton steps. The ARAP-GS cost is acceptable for editing but not for driving. Nodes are clustered into near-rigid parts by the residual of the ARAP fit: a truck cab and trailer separate, and a sedan stays one part. The static background is handled as in CRUISE, and FOCI's static constraint applies to it directly.

Bound. For a Gaussian attached at offset $r$ from its part's reference node, its mean over the window $[t_0, t_0 + \Delta]$ is bounded by a ball of radius

$$\rho = \|v\|\Delta + \tfrac{1}{2} a_{\max}\Delta^2 + \omega_{\max}\Delta\,\|r\| + \epsilon_{\text{arap}} + \epsilon_{\text{reg}}.$$

The symbols are defined as follows:

  • $\Delta$ is the sum of measured communication delay and planning horizon.
  • $v$ is the part's estimated velocity.
  • $a_{\max}$ and $\omega_{\max}$ are class-specific linear-acceleration and angular-rate limits, as in CIAO's action radius.
  • $\epsilon_{\text{arap}}$ is the largest non-rigid deviation of that part's nodes observed in the fit window.
  • $\epsilon_{\text{reg}}$ is the cross-agent extrinsic registration error, estimated from the disagreement between vehicle and infrastructure observations of the same static background.

The last term folds collaborative perception's registration-drift problem into the safety margin instead of ignoring it. Each Gaussian's covariance is inflated by $\rho^2 I$.

This bound is only as honest as $a_{\max}$, $\omega_{\max}$ and $\epsilon_{\text{arap}}$. It is a calibrated guarantee, not an unconditional one. Calibrating those quantities, and measuring how often reality exceeds them, is a central part of the work.

Planning. The ego trajectory is a cubic B-spline. Its MINVO control points per segment span a polytope $P_k$ that contains segment $k$. Safety for every inflated obstacle Gaussian with mean $\mu$ and covariance $\Sigma'$ active during that segment is imposed as the hard constraint

$$\min_{x \in P_k} (x-\mu)^\top \Sigma'^{-1} (x-\mu) \ge \kappa^2,$$

where $\kappa$ is the Mahalanobis radius at which a Gaussian counts as solid. The left side is a small convex quadratic program, and the constraint is linearized in a sequential convex scheme. Because the hull contains the whole continuous segment and the inflated ellipsoid contains the obstacle's whole window, no inter-sample tunneling is possible within the calibrated assumptions. Treating each Gaussian as a solid ellipsoid at radius $\kappa$ also closes the hollow-interior loophole the research file attributes to surface-only splats. We will check whether that makes the ego overly cautious around thin objects.

Baselines

Naive reference. Single-vehicle perception feeds a box-based planner that checks collisions at the 10 Hz frame rate with no latency compensation. This shows how much tunneling and how many missed occluded agents occur when nothing is done. All methods share the same ego planner and spline parameterization and differ only in obstacle representation, so differences cannot come from planner tuning.

Strongest competitor. This is the method a skeptical reviewer would actually deploy. Cooperative FF-Tracking boxes from V2X-Seq are extrapolated over the same window $\Delta$. Each box is inflated by the same CIAO-style action radius, using the same $a_{\max}$, $\omega_{\max}$ and $\epsilon_{\text{reg}}$, and enforced as hard constraints on the same MINVO hulls. This is the box version of our own method, given every advantage except fine geometry and rigidity structure. It is reported to maintain MOTA of 52.26 at 200 ms delay on V2X-Seq. For the box-based version, the TCLF projection from DAIR-V2X is the comparison point, with a reported gain of up to 1.5% AP under delays above 10 ms. No reported result was found on Lacuna for missed-occupancy or freezing metrics for either. We will compute these ourselves with identical inputs and identical tuning budget (the same grid over inflation and solver weights, selected on a held-out split). Beating it means equal or lower missed-occupancy at a significantly smaller swept volume, and lower freezing rate, on agents other than ordinary cars.

Continuous-time reference. FOCI is run with per-frame dynamic Gaussians re-inserted as static obstacles and its soft penalty at its published weighting. It is reported to solve in 0.5 to 2.1 seconds on scenes with over 138,000 Gaussians. This isolates the value of hard constraints and swept inflation over soft penalties on a snapshot.

Key ablation. This isolates the rigidity ingredient. With everything else fixed, we compare three ways of structuring each agent's Gaussians:

  • a single rigid cluster per agent, CRUISE-style;
  • free per-Gaussian velocities, Split4D-style, each Gaussian bounded independently;
  • the ARAP part graph.

A further ablation drops $\epsilon_{\text{reg}}$ to show how much safety depends on registration error.

Why not just…?

Why not just use cooperative boxes with an inflated margin? This is the real threat, and it is the strongest baseline above, configured as fairly as possible. For sedans in open lanes we expect it to tie, since a rigid box is nearly the true shape. Our claim lives in the cases where a box is a poor fit:

  • a trailer swinging through a turn;
  • a bus whose bounding box hides a free gap;
  • a cyclist whose swept box is mostly empty space.

FOCI's own comparison suggests shape-aware geometry matters: it found shorter paths than box-like planners because it modelled true orientation. That evidence comes from robots, not traffic. If the class-stratified results show no gain in conservatism on these agents, we will conclude that the Gaussian part graph is not worth its cost for driving. We would then restrict the claim to pedestrians and articulated vehicles, or abandon it.

Why not just check collisions densely in time? Dense sampling shrinks but never closes the gap between checks. The Implicit Swept Volume SDF work shows that discrete sampling suffers the tunnel effect in narrow passages. Cost also scales with the number of checks multiplied by the number of Gaussians. We will run the FOCI-style overlap at 10, 50 and 200 checks per second against the hull constraint. We will report both missed contacts found by an exact post-hoc continuous check and runtime.

Why not just give each Gaussian its own velocity bound, as Split4D's representation allows? This avoids building a deformation graph. However, independent bounds ignore that a vehicle's corner cannot move without its centre, so each blob receives the worst case. Split4D's boundary bleeding also means some Gaussians carry the wrong owner's velocity. The free-velocity arm of the ablation tests this directly. We expect it to give either larger swept volumes or more missed occupancy at contact zones.

Proposed Experiments

Open-loop bound validity. The main test uses V2X-Seq, because it pairs infrastructure and vehicle sequences with annotated tracks and is the setting where CRUISE reports reconstruction PSNR of 27.97. Validity is checked on DAIR-V2X, and on V2V4Real for vehicle-to-vehicle links. Communication delay is simulated at 0, 100 and 200 ms. Since V2V4Real reports that even 100 ms delay notably degrades performance, this range is where compensation matters.

For each agent and window we measure two things. Missed occupancy is the fraction of ground-truth future LiDAR points on that agent, within the annotated box, that fall outside the predicted swept bound. Conservatism is the ratio of bound volume to true swept volume. Both are stratified by class and by observed acceleration, so that abrupt manoeuvres are reported separately. Human-agent bound tightness is also measured on Split4D's Multi-Human multi-view data, where Split4D reports mIoU of 0.893. That data provides dense non-rigid ground truth that driving datasets lack. The calibrated limits are fit on training splits only.

Closed-loop safety. Ego trajectories are replayed in CRUISE-style reconstructed V2X-Seq intersections with edited agent trajectories that include sudden braking, turning trailers and crossing cyclists. Four quantities are reported:

  • collision rate under an exact continuous check against ground-truth geometry;
  • the subset of collisions a 10 Hz discrete check would miss, which counts tunneling directly;
  • freezing rate (the ego fails to make progress within a time limit);
  • planner latency.

Decisive checks. The proposal stands or falls on one comparison: the ARAP part-graph bound against the inflated cooperative-box bound on non-sedan agents at 100 to 200 ms delay. It must reach equal missed occupancy at clearly lower conservatism and freezing. A second check is that missed occupancy stays below a pre-set level, such as one percent of windows, when the calibrated limits are applied to held-out intersections. If that fails, the "guarantee" is not one, and we will report it as such. Finally, the per-frame refinement plus constraint solve must fit inside the planning cycle. If it cannot, the method is an offline safety auditor rather than a planner, and we will frame it that way.