A robot arm working in a kitchen or a lab has to know where it is allowed to move. Two common situations break the perception that answers this question. Depth cameras see through glass: they either return nothing on a transparent surface or report the wall behind it, so a wine glass shows up as empty space. Moving people and objects corrupt 3D reconstruction in the opposite direction, leaving phantom geometry behind. Methods that prove a motion is collision-free exist, but they take the obstacle map as given. They therefore inherit perception's worst error, which is false free space at exactly the places where the arm is most likely to break something.
This note proposes to stop trusting depth on these surfaces. The obstacle model would instead be built from an outer bound that the cameras can supply reliably. A transparent object is hard to range but easy to outline. Its outline in several images defines a carved volume, the visual hull, that must contain the object whenever the outlines are complete. Visual hulls are intersections of polyhedral cones. That makes them the polynomial-friendly shapes a proof tool needs. We would sample plausible hulls by perturbing the outlines, extrude moving objects along bounded velocities, and hand all of it to a sum-of-squares optimizer. Sum-of-squares (SOS) optimization proves a polynomial is nonnegative by writing it as a sum of squared polynomials, which reduces to a convex semidefinite program. Here it would certify regions of joint space that are collision-free against every sampled hypothesis. Held-out hypotheses then give a statistical safety level for the real, unknown geometry.
Core Claim
The core claim is that silhouette-derived outer bounds, combined with a scenario-style robust SOS program, yield joint-space regions with two properties at once. First, their collision rate against ground-truth transparent and moving geometry can be certified at a stated confidence. Second, they keep substantially more usable free volume than the obvious safe alternative of marking everything uncertain as occupied. What would be new is the obstacle model fed into the certificate, not the certificate machinery. The model is geometric and category-free, so it works zero-shot on unseen glassware. It does not depend on depth returns from transparent surfaces. Its uncertainty is explicit, so a probability-of-safety statement can be attached.
Background and Positioning
The certification engine is reused from C-IRIS (Finding and Optimizing Certified, Collision-Free Regions in Configuration Space for Robot Manipulators). C-IRIS rewrites joint angles $q_i$ through $s_i=\tan(q_i/2)$ so that forward kinematics become rational polynomials. It then alternates two steps: fitting an inscribed ellipsoid while SOS-certifying separating planes between each link and each convex obstacle, and pushing polytope faces outward. It filled about 90% of the free configuration space of a 3-DOF system with 11 regions and scaled to a 7-DOF KUKA iiwa at a shelf. Its stated limitation motivates this work: obstacles must be known, static, convex semi-algebraic sets, and raw sensor streams cannot enter. Visual hulls and swept hulls are convex polyhedra, or unions of a few of them, so they are drop-in inputs. Uncertainty enters through sampled hypotheses rather than through a new type of constraint.
The uncertainty treatment borrows from Barrier Certificates for Unknown Systems with Latent States and Polynomial Dynamics using Bayesian Inference. That paper draws posterior samples of uncertain dynamics, solves one robust SOS program over a training ensemble, and validates on held-out samples with a Clopper-Pearson bound. It reports a mean safety probability of 98.75% at 99.9% confidence. We transplant the idea from uncertain dynamics to uncertain geometry. The samples are obstacle hypotheses instead of model parameters, and the certificate is a separating-plane family instead of a barrier function.
For speed, the scaled-diagonally-dominant relaxation of Some Applications of Polynomial Optimization in Operations Research and Real-Time Decision Making is the natural fallback. It replaces the semidefinite constraint with second-order cones and synthesized Dubins-car barrier certificates in 20 to 30 ms. Scenario ensembles multiply the constraint count, so this relaxation is likely necessary.
The dynamic-scene literature constrains the design more than it supplies it. RobustSplat shows why transients produce persistent floaters: Gaussian splatting grows new primitives wherever photometric error spikes, and transients cause exactly such spikes. It mitigates this with delayed densification and coarse-to-fine transient masks. Split4D assigns each Gaussian a linear velocity and an identity feature, which decomposes a scene into moving entities. Both target image fidelity. For safety the asymmetry is reversed. A floater is a harmless false obstacle that only costs free volume, while a removed transient is a real moving object the arm must still avoid. We would use their transient masks and per-entity velocities only to build time-extruded hulls. We would not use them to erase anything.
On the geometric side, GeoUDF fits local quadratic patches to point neighbourhoods and reconstructs open surfaces zero-shot. It reports a Chamfer distance of 0.194 on unseen garments after training only on watertight shapes. We would use it for the opaque, well-ranged part of the scene and convexify its output. The dossier itself notes that such distance fields collapse when depth drops out on refractive surfaces, which is why transparent regions take the silhouette path.
The research file I had was truncated before its survey of existing work at this exact intersection. I therefore cannot rule out a prior system that combines visual hulls with SOS certification. Nothing in the material I saw does so.
Research Program
The pipeline has three stages.
The first stage builds hypotheses. We would take a calibrated multi-view RGB-D capture of a tabletop scene. Opaque geometry comes from depth, reconstructed with GeoUDF and decomposed into convex pieces. Transparent objects come from per-view segmentation outlines intersected into a visual hull. To form $N$ hypotheses $\{\mathcal{O}^{(k)}\}$, we would dilate and erode each outline by amounts drawn from an error model fit on held-out annotated views, and resample which views participate. Moving entities come from RobustSplat-style transient masks or Split4D velocities. Each becomes its hull swept over the planning horizon $T$, using a velocity bound $\bar v$ inflated by an estimated error.
The second stage is robust certification. For a candidate polytope $P$ in the stereographic coordinates $s$, robot links $\mathcal{L}_j$, and every obstacle piece $o$ in every training hypothesis, we search for a separating plane whose coefficients are polynomial in $s$. Specifically, we seek $a_{j,o}(s)$ and $b_{j,o}(s)$ such that
$$a_{j,o}(s)^\top x + b_{j,o}(s) \ge 1 \;\; \forall x \in \mathcal{L}_j(s), \qquad a_{j,o}(s)^\top y + b_{j,o}(s) \le -1 \;\; \forall y \in o, \qquad \forall s \in P.$$
Each implication becomes an SOS constraint through the standard multiplier construction C-IRIS uses. Many hypotheses share most of their pieces: only the transparent and transient parts vary. Constraints for shared pieces are therefore written once, which keeps the program size close to linear in the number of uncertain objects rather than in $N$.
The third stage validates. On $M$ held-out hypotheses we would count the regions that are violated. Clopper-Pearson then turns that count into a lower bound on the probability that the true scene, treated as a draw from the hypothesis distribution, is collision-free throughout the region. The weak link is the assumption that real outline errors follow the error model. The experiments test this directly against ground truth rather than assuming it.
Baselines
The naive reference is C-IRIS run directly on obstacles reconstructed from raw depth. Glass appears missing or wrong there, and transients appear as floaters. All other methods use the same robot, scene, seeds and iteration budget. We expect this baseline to keep the most free volume while frequently intersecting real glass. Its violation rate defines the problem.
The strongest safe competitor is unknown-as-occupied. It runs C-IRIS on a voxel map where every voxel not positively carved free by depth rays counts as an obstacle, and it adds a fixed safety margin. This is the standard conservative practice. It needs no learning and is the alternative a reviewer would reach for. We would tune its margin on the same validation scenes we use to fit our outline error model, to the margin that minimizes violations. Beating it means matching or lowering its ground-truth violation rate while retaining more certified free volume, measured as the fraction of sampled collision-free configurations covered by a fixed number of regions.
A third baseline gives C-IRIS its best available obstacle map. It reconstructs a deterministic transparent shape, for example a single visual hull without sampling or a learned transparent-depth completion, and certifies against it. I found no reported result on Lacuna for transparent-object depth completion accuracy or for its use in certified planning. Its numbers would come entirely from our runs.
Two ablations isolate the key ingredients. One keeps a single deterministic hull to test whether hypothesis sampling earns its cost. The other drops time-extrusion for transients to test whether erasing them, as fidelity-oriented methods do, is unsafe. For transient-filtered reconstruction quality itself, RobustSplat reports beating SpotLessSplats, WildGaussians and T-3DGS on PSNR, SSIM and LPIPS on NeRF On-the-go and RobustNeRF. The file gives no numeric values, so no reported figure is available to anchor against.
Why not just…?
Why not just mark all unobserved space as occupied? This only protects you from unknowns that you know are unknown. Depth sensors usually return the surface behind the glass, so rays pass through the glass and carve it as free. Unknown-as-occupied then fails in the same way the naive baseline does, unless the margin is inflated until most of the table is lost. This is the real threat to the proposal, and the margin-tuned comparison above settles it. If a tuned margin matches our violation rate at similar free volume, the claim fails.
Why not just complete the transparent depth with a learned network and plan as usual? A completion gives one best guess with no outer-bound property. Its errors are largest on unusual shapes, and those are exactly where zero-shot claims matter. The deterministic-completion baseline measures this directly. If it achieves low violation rates on unseen glassware, the visual-hull machinery is unnecessary for transparency and the contribution narrows to transients.
Why not just add a safety margin around everything and skip certification, using dense sampled collision checks? Sampling-based checks can miss thin regions and give no region-level statement. Separately, the margin-only approach still inherits false free space on glass. We would report both certified volume and empirical collisions from dense sampling inside each certified region, so readers can see whether the certificate adds anything beyond the margin.
Proposed Experiments
We would use a 7-DOF arm reaching into a cluttered shelf, mirroring C-IRIS's KUKA iiwa setting. The shelf holds unseen glassware: tumblers, flutes, beakers, and irregular decorative glass. Some trials add a person's hand or a rolled object moving through the workspace. Ground-truth geometry would come from scanning each glass item after temporarily coating it opaque and registering the scan back into the scene. Moving objects would be motion-captured. The dynamic-capture component would also be evaluated offline on NeRF On-the-go and RobustNeRF sequences to calibrate transient masks. Those datasets lack robot ground truth, so they test hypothesis construction rather than safety.
We would measure three things. The first is the ground-truth violation rate, the fraction of certified regions that intersect true geometry, checked densely. The second is certified free-volume coverage. The third is the certificate's claimed probability against the empirically observed one. A stated level such as 98% must not exceed the observed safe fraction across real scenes. Solve time per region under full SOS versus the scaled-diagonally-dominant relaxation is the fourth number. It matters if regions are to be refreshed as transients move.
The decisive check is the comparison with tuned unknown-as-occupied on held-out glass shapes never used to fit the outline error model. The proposal succeeds if it reaches comparable or lower violation rates with clearly higher coverage, and if its claimed safety probability is not optimistic relative to the observed rate. It fails informatively if the bound is systematically optimistic. That would show outline errors on real glass are correlated across views in ways independent perturbation misses, and the hypothesis generator would be the next thing to fix.
References
- Finding and Optimizing Certified, Collision-Free Regions in Configuration Space for Robot Manipulators
- Barrier Certificates for Unknown Systems with Latent States and Polynomial Dynamics using Bayesian Inference
- Some applications of polynomial optimization in operations research and real-time decision making
- RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS
- Split4D: Decomposed 4D Scene Reconstruction Without Video Segmentation
- GeoUDF: Surface Reconstruction from 3D Point Clouds via Geometry-guided Distance Representation