A robot that sees an object from one camera angle has to guess the back half. Several grasping systems now make that guess with a shape-completion network run under Monte Carlo dropout. Monte Carlo dropout keeps the network's random neuron-dropping switched on at test time, so repeated forward passes return a spread of plausible shapes instead of a single one. These systems then score each candidate grasp by how well it does across that spread. The approach works, but the probabilities it produces are never checked against reality. A single-view shape posterior tends to be least reliable on the hidden side, which is exactly where the fingers usually land. As a result, "this grasp holds on 90% of sampled shapes" may have little to do with how often it actually holds.
We propose to make the probability honest before acting on it. The robot takes a small number of light probing touches. It reweights and mutates the set of dropout masks so that only shape hypotheses consistent with those touches survive, using a particle filter over masks as in Adapting Neural Models with Sequential Monte Carlo Dropout. It then commits only to a grasp whose probability of force closure and collision-free execution, bounded from below with a confidence interval, clears a stated threshold. The claim is concrete. After a few touches, the dropout posterior's predicted feasibility probabilities become calibrated, meaning they match simulated outcomes. That calibration is what turns "chance constraint" from a label into a guarantee the planner can use.
Where this sits
Two papers already combine Monte Carlo dropout with probabilistic grasp evaluation, and they supply this proposal's starting point. Robust Grasp Planning Over Uncertain Shape Completions samples voxel completions with dropout and averages Ferrari-Canny grasp quality over the samples. On hardware it reports 59% success against 48% for planning on a single completion. Measuring Uncertainty in Shape Completion to Improve Grasp Quality runs 60 dropout passes through a transformer completion model and subtracts a hand-weighted variance penalty from grasp scores. This lifts GPD from 35% to 58% and PointNetGPD from 18% to 49% on YCB objects. Both treat the dropout spread as a fixed snapshot of a single view. Neither asks whether its sample frequencies are calibrated. Neither turns them into a threshold with a stated failure rate. Our proposal differs in all three respects: it conditions the posterior on contact, it measures calibration directly, and it enforces a lower-confidence-bound chance constraint instead of a mean or a penalty.
The sequential Monte Carlo dropout paper supplies the key mechanism. It treats binary dropout masks as a latent variable, keeps weights frozen, and tracks a particle distribution over masks with bit-flip mutations. It was used only for dynamics adaptation on a Panda arm, never for shape perception. We reuse the technique and change the likelihood to a geometric one.
The analytic robustness work in Toward An Analytic Theory of Intrinsic Robustness for Dexterous Grasping motivates the constraint side. Its probabilistic surface normals yield a lower bound on the probability of force closure, but those normal beliefs are hand-specified rather than learned. Our particle set gives an empirical, learned counterpart. GRaCE shows that phrasing collision and stability as probabilities rather than binary filters pays off in clutter, reporting 65% against 31% on hardware, but it does not say where those probabilities come from.
Gaussian process surfaces updated by touch, as in Simultaneous Tactile Exploration and Grasp Refinement for Unknown Objects and SpringGrasp, are the principal competitor rather than a precursor. They already fuse contact natively. What they cannot do is draw on category-level shape knowledge learned from thousands of objects to fill in unseen parts from one view.
Core mechanism
Let $x$ be the single depth view. A dropout mask $m$ induces a completed shape $S_m = f(x; m)$ with signed distance function $d_m(\cdot)$. We start with $N$ particles $\{m_i\}$ drawn from the training dropout distribution. After the robot records contact points $c_1,\dots,c_K$, it also knows the free-space points $F$ that the probing finger swept through without touching anything. Each particle's weight is updated as
$$w_i \propto w_i \,\exp\!\Big(-\sum_{k=1}^{K} \frac{d_{m_i}(c_k)^2}{2\sigma_c^2}\Big)\prod_{p \in F} \mathbf{1}[d_{m_i}(p) > 0].$$
Here $\sigma_c$ is the contact sensing noise. The indicator discards hypotheses that place material where the finger passed freely. When effective sample size collapses, we resample and apply bit-flip mutations to explore masks near the survivors.
For a candidate grasp $g$, let $\hat p(g)$ be the weighted fraction of particles on which $g$ both achieves force closure (positive Ferrari-Canny margin) and collides with nothing. We compute a one-sided lower bound $\underline{p}(g)$ from $\hat p(g)$ using the Kish effective sample size and a Clopper-Pearson interval. The planner then selects
$$g^\star = \arg\max_g \; \mathbb{E}_w[Q(g; S_m)] \quad \text{subject to} \quad \underline{p}(g) \ge 1-\delta.$$
In this rule, $Q$ is grasp quality and $\delta$ is the tolerated failure rate. Probes are placed where particles disagree most about occupancy. If no grasp clears the bound within the probe budget, the planner reports that no grasp is safe rather than guessing.
Research Program
The work has three parts, and each can fail informatively.
The first part is a calibration study with zero touches. We train a voxel completion model in the style of Lundell et al. and a point-cloud transformer in the style of 3DSGrasp, both with dropout, on rendered single views of YCB meshes (Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols) plus the Dex-Net 2.0 CAD set. For thousands of grasps, we compare predicted feasibility probability against the success of the same grasp on the true mesh in physics simulation. Separately, we score calibration on the camera-facing side and on the hidden side. The hypothesis is that hidden-side calibration is poor and that tuning the dropout rate alone cannot fix it. The file notes that dropout calibration depends heavily on the rate, which is usually set as a static hyperparameter.
The second part builds the touch-conditioned particle filter and repeats the calibration study at 1, 2, and 3 probes. This is where the method's central claim is tested.
The third part closes the loop with chance-constrained selection. We test whether the realized failure rate of committed grasps stays at or below $\delta$ across a sweep of $\delta$, and how success and time compare with the baselines.
Parallel-jaw grasping is the main setting. Multi-fingered force closure on the Allegro hand model is a secondary test, since it is the regime the analytic robustness work targets.
Baselines
The naive reference is deterministic planning on a single completion, with collision geometry padded by a tuned safety margin. The margin is tuned on validation objects for success rate, so that padding gets its best chance rather than a straw-man setting.
The strongest existing Monte Carlo dropout method is the sample-averaged robust planner of Robust Grasp Planning Over Uncertain Shape Completions. Its reported number is 59% hardware grasp success on 10 objects, against 48% for a single-completion baseline. We also include the variance-penalized re-ranking of Measuring Uncertainty in Shape Completion to Improve Grasp Quality, which reports 58% with GPD. Both are re-run in our simulator on the same completion networks, the same candidate grasps, and the same forward-pass budget. Their hardware figures are cited only as context, because those setups differ from ours.
The strongest touch-based competitor is a Gaussian process implicit surface fused with the same probe contacts under the same probe budget. We give it the identical chance-constrained selection rule, so any difference comes from the shape posterior and not from the decision rule.
A deep ensemble of five completion networks, reweighted by the same contact likelihood, tests whether a small discrete ensemble does as well as the dropout mask space. It is matched for inference compute. Because an ensemble costs several times more to train, we report training cost alongside its results.
Three ablations isolate the proposal's ingredients. The first reweights a fixed set of $T$ dropout samples by contact with no resampling or mutation. The second applies the chance constraint with zero probes. The third replaces the lower confidence bound with the plain weighted mean.
Beating the baselines requires three things together. Hidden-side calibration error must be clearly lower than every baseline at matched probe count. The realized violation rate must stay within $\delta$ where the mean-based rules exceed it. Simulated success must be at least that of the Gaussian process baseline on objects with parts that cannot be inferred from the visible surface.
No reported result was found on Lacuna for grasp-feasibility calibration error or for realized chance-constraint violation rate under any of these methods. Those comparisons will be established here.
Why not just…?
Why not just average over dropout samples, as Lundell et al. do? Averaging ranks grasps sensibly but does not state how often a chosen grasp will fail. If hidden-side samples are overconfident, the average inherits that error. The zero-probe calibration study measures this directly. If single-view averages turn out to be already calibrated, then touching adds little and the contribution shrinks to the confidence-bound rule. We will report that outcome as it falls.
Why not just reweight a fixed batch of dropout samples by contact? This is the most serious simpler alternative, and it is one of our ablations. With 60 samples and even two informative contacts, we expect most weight to collapse onto a few samples. The bound on $\underline{p}$ then becomes too loose to certify any grasp. Bit-flip mutation, following Carreno et al., is what restores diversity. If fixed-batch reweighting matches the full filter on calibration and success, we will drop the filter and recommend the simpler method.
Why not just use a Gaussian process surface, which already fuses touch? Gaussian process surfaces give closed-form variance and update natively on contact. The file notes that their cost grows cubically with the number of observations and that they cannot draw on learned category shapes. The threat is real on simple convex objects, where we expect the Gaussian process to tie or win. The deciding evidence is the split of results by object type: our method should win on objects with handles or appendages hidden from the camera.
Why not just take a second camera view? SpringGrasp reports 89% success with two views against 84% with one, so extra views help. Many placements, such as objects against walls or in bins, do not allow a second view, while a probe touch is available. Where a second view is feasible, we include it as a reference at matched acquisition time.
Decisive checks
The first decisive check is the hidden-side reliability diagram before and after 1 to 3 probes. The second is realized failure rate plotted against $\delta$ for our method and for the mean-based rules. The third is simulated success broken down by object type against the Gaussian process baseline, under an equal probe budget. The method succeeds only if all three favor it. If calibration improves but the bound never certifies grasps within a practical time budget, we will report that as a negative result on sample efficiency.
References
- Adapting Neural Models with Sequential Monte Carlo Dropout
- Robust Grasp Planning Over Uncertain Shape Completions
- Measuring Uncertainty in Shape Completion to Improve Grasp Quality
- Toward An Analytic Theory of Intrinsic Robustness for Dexterous Grasping
- GRaCE: Balancing Multiple Criteria to Achieve Stable, Collision-Free, and Functional Grasps
- Simultaneous Tactile Exploration and Grasp Refinement for Unknown Objects
- SpringGrasp: Synthesizing Compliant, Dexterous Grasps under Shape Uncertainty
- Benchmarking in Manipulation Research: Using the Yale-CMU-Berkeley Object and Model Set