Smoothing by Measured Noise: Identifying Contact Physics from Noisy Real Pushes

When a robot pushes an object, the outcome jumps abruptly depending on whether, where, and how hard contact happens. Simulators that can be differentiated, so they report how outcomes change as physical parameters change, should in principle let us fit friction and mass from a handful of real pushes. In practice, two problems get in the way. Contact makes the gradients useless: they are zero before contact and explode at impact. Real observations are also noisy. The standard fix for the first problem is randomized smoothing, which averages the simulator over a cloud of perturbed inputs. That fix brings its own question: how wide should the cloud be? Too narrow, and the gradients stay blind. Too wide, and the fitted parameters describe a blurred world rather than the real one.

This note proposes that the width should not be tuned at all. A real robot already executes its commands with measurable error, and its cameras already report object poses with measurable error. If the injected perturbations are drawn from that calibrated noise, the smoothed simulator is no longer an approximation. It becomes the correct statistical model of the noisy data, so the averaging that makes gradients informative also makes the fitted parameters unbiased. The second commitment is to inject that noise in the robot's command space and pass it through the robot's kinematics, not into object or pusher positions. Every sampled rollout is then physically executable, and none starts with the pusher embedded inside the object.

Core Claim

Let $\theta$ be the unknown physical parameters, namely the friction coefficient, the center of mass, and mass. Let $u_{1:T}$ be the commanded pusher motion and $y_{1:T}$ the observed object poses. The model assumes execution noise $\epsilon \sim p_{\text{act}}$ in command space, observation noise with covariance $\Sigma_y$, and a deterministic contact simulator $s_\theta$ composed with the pusher's forward kinematics $k$. Under these assumptions, the physically meaningful fitting objective is the marginal likelihood

$$ \mathcal{L}(\theta) = -\log \, \mathbb{E}_{\epsilon \sim p_{\text{act}}}\Big[\mathcal{N}\big(y_{1:T};\; s_\theta(k(u_{1:T}+\epsilon)),\; \Sigma_y\big)\Big]. $$

A Monte Carlo estimate of this expectation and its gradient is exactly a gradient bundle in the sense of Bundled Gradients through Contact via Randomized Smoothing. The difference is that the smoothing distribution is the calibrated $p_{\text{act}}$, not a hand-set Gaussian.

The claim is falsifiable in three parts. First, identification with a deterministic simulator smoothed at an arbitrary width $\sigma$ should show parameter bias that grows with distance from the calibrated noise level. Second, matched-noise smoothing should reach parameter error at least as low as the best grid-searched width, without any search. Third, command-space injection should beat position-space injection at equal sample count, because position-space samples produce interpenetration and artificial penalty forces.

Relation to Prior Work

Prior work supplies the method. Suh, Pang and Tedrake showed that bundled gradients let trajectory optimizers in Drake "see" contact at a distance, reporting a 36x speedup over penalty formulations with $N=100$ samples per knot point. However, they validated only in simulation with perfect state and chose the smoothing as an optimization device. Do Differentiable Simulators Give Better Policy Gradients? explains why exact contact gradients carry empirical bias. It proposes the α-order estimator, which blends exact gradients with sampling-based gradients under a confidence test whose thresholds the file flags as stiffness-sensitive. We reuse the estimator family but replace the tuned knobs with a measured noise model.

Prior work also motivates the proposal. D-REX backpropagates real-versus-simulated push discrepancies into object mass, reaching 4.8%–12.0% mass error. It assumes clean segmentation and identifies mass only, leaving friction and center of mass, which are what actually govern pushing, unaddressed. We target exactly those parameters under deliberately noisy perception.

Two papers supply the key analogy. Evaluating randomized smoothing as a defense against adversarial attacks in trajectory prediction found that noise added to vehicle controls and integrated through a bicycle model beats noise added to raw coordinates. On L-GAP under the strongest attack, error fell from 2.952m to 1.428m. That work had no contact. Similarly, Trajectron++ and ControlMTR show that routing outputs through kinematic models removes infeasible motion. We test whether the same command-space principle carries over to contact, where infeasible samples are not merely implausible but produce wrong forces.

One paper constrains the design. QuasiSim obtains smooth contact landscapes by relaxing the hand into unconstrained point sets and softening contact. It reports 8/12 real Allegro successes, against 4/12 for DGrasp. Its intermediate stages violate kinematics, and the file notes that policies exploit softening artifacts. Our design keeps rigid-ish contact and full kinematics and gets smoothness from noise rather than from relaxed physics.

Research Program

The testbed is Omnipush, which contains 62,500 real pushes on 250 modular objects with systematically varied shape and internal mass distribution. Its most useful property is that every push is recorded twice. Vicon cameras at 250Hz give near-ground-truth object poses, and a RealSense RGB-D camera at 30Hz gives realistic noisy poses. We can therefore fit parameters from the noisy stream and score them against the clean one. The object construction also gives approximate ground truth for center of mass, from the known 60g and 150g weight placements.

We will build three components. The first is a noise calibration stage. It estimates $p_{\text{act}}$ from the spread of Vicon-measured pusher motion around commanded motion, and it estimates $\Sigma_y$ from RGB-D-tracked poses against Vicon on held-out pushes. The second is a planar quasi-dynamic pushing simulator in a differentiable engine. The pusher is driven through its kinematic chain, so command perturbations become pusher motion only via $k$. The third is the matched-noise bundled estimator of $\nabla\mathcal{L}$. It will be run with first-order gradients through each sample, and also as a sampling-only (zeroth-order) estimate, so that the contribution of the objective can be separated from that of the gradient type.

The analysis has two parts. The first sweeps the smoothing width from zero to well above the calibrated noise and records parameter bias against Vicon-derived fits, giving the bias-versus-width curve the core claim predicts. The second makes the noise heavier by corrupting RGB-D tracks further, with frame drops and added depth jitter, to see whether matched smoothing tracks the changing noise automatically while fixed-width smoothing degrades.

Baselines

The naive reference is fitting with exact, unsmoothed gradients through the differentiable simulator, initialized from no-contact guesses, in the style of D-REX's gradient-based identification. This is what a practitioner would try first. The file reports that exact-gradient iMPC "failed to make progress from non-contact initial guesses," so we expect it to stall. D-REX's 4.8%–12.0% mass error is the only reported identification number in the file. It was obtained on different objects and with clean segmentation, so it is a reference scale, not a head-to-head comparison. No reported result was found on Lacuna for differentiable friction or center-of-mass identification on Omnipush.

The strongest existing competitor is Suh-style isotropic Gaussian bundling with the width chosen by grid search on a validation split of pushes. It directly embodies "smoothing is a tunable optimization trick." It will receive the same $N=100$ samples per evaluation and the same number of optimizer iterations. Its grid search costs extra compute, which we report but do not charge against it. Beating it means matching or lowering parameter error, and held-out Vicon-trajectory prediction error, with zero tuning. If grid search wins clearly, the core claim fails.

The α-order estimator from Suh et al. is run with its published interpolation protocol at its default thresholds and with the same sample budget. It represents adaptive handling of bad gradients without a noise model. Note that the file reports only qualitative wins for it on Tennis, Ball-with-Wall and pushing.

A non-identifying reference is domain randomization over friction and mass, which plans with the whole range. D-REX found this kind of approach reaches 40% grasp success, against 90% with identified mass, on a 324g object.

A learned non-physical reference is the Attentive Neural Process from Omnipush. The file reports that it halved the distance to the Bayes error rate with 50 seed pushes, but gives no absolute number, so we retrain it on our splits.

Two ablations isolate the key ingredients. One keeps matched-magnitude noise but injects it in pusher position space instead of command space; this isolates the kinematic routing. The other keeps command-space injection but uses an isotropic Gaussian of matched total variance; this isolates the noise shape.

Why not just…?

Why not just use a derivative-free optimizer like the cross-entropy method on the handful of parameters? This is a real threat. With roughly four to six parameters, black-box search may fit just as well, and Suh et al.'s advantage over that method was shown on high-dimensional trajectory optimization, not low-dimensional identification. Our response is to make the claim about the objective rather than the optimizer. We will run the cross-entropy method on both the matched-noise likelihood and the deterministic squared-error loss. If the matched objective improves the cross-entropy method's parameter bias too, the contribution stands independent of gradients. The gradient estimator's added value is then tested by scaling to a spatially varying friction field over the object footprint, where black-box search should degrade.

Why not just tune the smoothing width on validation data? This is the strongest baseline above. It may fit equally well on Omnipush's nominal noise. The predicted difference shows up when noise conditions shift, for example with heavier RGB-D corruption or a different camera. In that case a tuned width goes stale, while a recalibrated noise model costs a few dozen Vicon-labeled pushes rather than a fresh grid search. If tuning remains competitive under shift, we will report matched noise as a principled default rather than an accuracy gain.

Why not just skip physics and learn the push dynamics directly? The Omnipush Attentive Neural Process already predicts pushes well from 50 seed pushes, and it may beat us on in-distribution prediction error. It does not, however, produce friction or center-of-mass values that a planner or a different robot can reuse. Our test is therefore extrapolation: predicting pushes with new pusher contact points and speeds outside the seed set, where a physical model with correct parameters should generalize and a learned predictor should not.

Proposed Experiments

The primary measures are three: parameter error against Vicon-fitted and construction-derived ground truth; average displacement error of predicted object trajectories on held-out real pushes, scored against Vicon; and the fraction of optimization runs that leave the no-contact plateau. These are measured on a split of Omnipush objects, across nominal and degraded RGB-D noise, for every baseline and ablation.

The first decisive check is the bias-versus-width curve. If its minimum does not fall near the calibrated noise width, the central idea is wrong. The second is the command-versus-position ablation. We expect position-space samples to show penetration-driven force spikes and higher gradient variance at equal $N$; if they do not, the kinematic routing is unnecessary. The third, conditional on hardware access, is closed-loop: a sampling-based push-to-pose controller planning with each method's identified parameters on a physical pusher, compared on success rate within a fixed pose tolerance. Without hardware, the extrapolation test on held-out pusher configurations stands in for transfer.