When the Simulator Assumes Symmetry: Real-Data Budgets for Singular Calibration Parameters

Physics simulators used to evaluate robots usually make small symmetric assumptions. They place a link's center of mass on its axis, align joint axes exactly, give friction the same value in both directions, and model noise as symmetric. Real hardware breaks each of these a little. The parameters that describe those small departures are a large part of the reality gap: the mismatch between how a robot behaves in simulation and how it behaves physically. This note argues that such parameters are statistically special at the simulator's default value. Because a mirror-image departure (an offset of $+\theta$ or $-\theta$) produces the same trajectory to first order, the data carry no first-order information about them there. Their Fisher information, which measures how sharply the data pin a parameter down, is singular at that point. As a result, the amount of real data needed to calibrate them, or to certify that a simulator matches reality, grows like $\epsilon^{-4}$ in the target accuracy instead of the usual $\epsilon^{-2}$.

We propose to test this claim directly. If it holds, two things follow. First, it explains part of the "parameter collapse" that high-dimensional simulator tuning keeps running into. Second, it gives a practical recipe: find the symmetry-singular directions, then design real-world test motions that break the symmetry and so restore the fast rate. The idea transfers a mechanism that is well understood in mathematical statistics, the Fisher-information singularity of skew-symmetric models at exact symmetry, to the question of how much physical validation a simulator-based evaluation actually needs.

Core Claim

Calibrating a simulator to real data is parameter estimation. Two empirical observations show it degrades as the parameter count grows. Traversing the Reality Gap via Simulator Tuning tuned physics-engine parameters with Differential Evolution against Kinova arm trajectories and found that moving from 31 shared to 57 individual parameters added noise rather than accuracy. DEAL reports "parameter collapse" of classical system identification at up to 125 parameters and replaces it with a diffusion-guided evolutionary search. Neither paper explains the collapse. These two papers motivate the work.

The mechanism comes from statistics. Ley's score-function paper shows that for the skew-normal at symmetry, the skewness score becomes proportional to the location score. The score is the gradient of the log-likelihood with respect to a parameter. When two scores point the same way, the likelihood cannot tell the two parameters apart. Ley and Paindaveine and Hallin and Ley show this happens for whole families of models, not just the normal, and that it produces flat profile likelihoods that stall optimizers. Brunel shows the version that matters most here. When the true parameter is fixed by a symmetry group, the Fisher information loses rank. Estimation then splits into two speeds: $n^{-1/2}$ in directions that break the symmetry and $n^{-1/4}$ along the symmetric directions. These papers supply the method. They study only distributions and finite groups, never dynamical simulators.

Our claim is that symmetry-breaking simulator parameters fall into Brunel's slow class at their default value. Write $\theta$ for such a parameter, for example the lateral center-of-mass offset of a pushed cube, a joint-axis tilt, or a friction asymmetry, with $\theta = 0$ the simulator's symmetric default. If the simulated trajectory is unchanged under $\theta \to -\theta$, then its expected squared deviation from the default trajectory behaves as

$$ D(\theta) = c\,\theta^4 + O(\theta^6), $$

where $D$ is that expected squared deviation, measured relative to the noise level, and $c$ depends on the test motion. The quartic form arises because the trajectory itself moves only at order $\theta^2$. Averaging over $n$ trajectories therefore estimates $\theta^2$ at rate $n^{-1/2}$, which means $\theta$ is estimated at rate $n^{-1/4}$. A test motion that breaks the mirror symmetry, such as pushing the cube off its centerline, adds a $\theta^2$ term to $D$ and restores the $n^{-1/2}$ rate.

The pushing failures reported in Quantifying the Reality Gap in Robotic Manipulation Tasks are suggestive here. MuJoCo and Bullet 2.83 introduced a spurious 90-degree pitch on a pushed cube that barely rotated in reality. Centered pushes of a near-symmetric object are exactly where these parameters are least identifiable.

Research Program

The first part builds a diagnostic that sorts each direction in parameter space into one of three classes. A regular direction has a well-conditioned Fisher information. A symmetry-singular direction has zero second-order information but nonzero fourth-order information, and so converges at the slow rate. A flat direction is unidentifiable at every order. The diagnostic computes score covariances from simulator rollouts by finite differences, or by autodiff where the engine allows it. For the null directions of that covariance, it fits the quartic coefficient $c$ along each direction.

This third class is what separates the work from ordinary sensitivity or "sloppiness" analysis. Those methods stop at second order and would label symmetry-singular directions as flat.

The second part designs excitations. Among a library of feasible test motions on the real robot, we choose those that maximize the second-order information along directions the diagnostic flagged as symmetry-singular. This is the analogue of moving the true parameter away from the singular point. We also predict, for any given motion set, how many real trajectories are needed to certify each parameter to tolerance $\epsilon$.

The third part reports uncertainty for calibrated simulators that stays honest under singularity. Standard intervals assume $n^{-1/2}$ behavior. We instead use profile-likelihood intervals and the Bayesian distance-based priors surveyed in Bayesian Inference for Skew-Symmetric Distributions, which were built to handle flat likelihoods at symmetry. We check their coverage empirically.

Baselines

The naive reference point is the simulator with default parameters and no calibration. It is the floor that both Collins et al. and the Kinova Mico2 study measure against. Collins et al. report trajectory-accuracy gains of 14% to 91% from tuning over this default.

The first strong competitor is Differential Evolution (best1bin) on Euclidean trajectory error, exactly as in Collins et al. It is the most widely reproduced black-box tuner, and it is the method whose collapse at 57 parameters motivates this work. We rerun it on the same trajectories with the same simulator-call budget as our method.

The strongest existing method is DEAL, which combines a real-versus-simulated trajectory discriminator with diffusion-style evolutionary search. It is the only method in the file shown to calibrate up to 125 parameters. No reported search-error result was found on Lacuna for DEAL, so we will rerun it ourselves in its sim-to-sim setting with matched simulator calls and tuning budget.

On REALM, the reference is the calibrated 14-parameter IsaacSim model of REALM. It reports a Pearson correlation of 0.921 between simulated and real task progression.

The key ablation is "second-order design": excitations chosen by D-optimal design, which maximizes the determinant of the Fisher information at the default parameters, with no fourth-order analysis. A second ablation is our diagnostic without excitation redesign.

Beating a baseline means one of two things. Either we reach the same parameter error with materially fewer real trajectories, which is the quantity our rates predict, or we reach lower held-out trajectory error on symmetry-breaking test motions at equal data. On REALM, it means matching or exceeding 0.921 on the stress tests that move objects off-center.

Why not just…?

Why not just use standard sensitivity or sloppiness analysis (eigenvalues of the Jacobian-squared)? This is the most serious threat. For regular directions, our diagnostic reduces to exactly that analysis. Our contribution is confined to the null directions, where second-order analysis cannot tell "slow but learnable" from "never learnable." The sim-to-sim experiments settle it. If second-order analysis predicts the real-data budget for planted center-of-mass offsets as accurately as the fourth-order prediction does, the proposal reduces to a known tool, and we will say so.

Why not just run D-optimal excitation design and remove the singularity? For many parameters this may be enough. That is why it is an ablation and not a strawman. The difficulty is that at a symmetric default, the second-order information along a symmetric direction is zero for every centered motion. The determinant criterion therefore gives no gradient toward symmetry-breaking motions unless they are already in the library. Brunel's analysis says the fix is to break the stabilizer, not to rescale. If D-optimal design finds the same motions, the excitation part of our program is redundant, but the rate predictions still stand.

Why not just use domain randomization and skip calibration? DEAL reports that domain-randomized Go2 controllers "frequently failed or drifted significantly." RRLS shows that worst-case robust agents give up nominal performance. More importantly, randomization does not answer the evaluation question this note targets: how much real data certifies that a simulator's verdict on a policy is trustworthy.

Why not just learn a neural residual on top of the simulator? Neural-augmented simulation corrected PyBullet with an LSTM trained on about 500 real episodes. However, it is accurate only where the data went, and it needs per-step state resets. A residual network does not remove the identifiability problem; it absorbs it. If the data never break the symmetry, the residual cannot learn the asymmetric effect either.

Proposed Experiments

The decisive test is sim-to-sim, where ground truth is known. In RRLS MuJoCo environments, and in a pushing scene modeled on the Mico2 cube task, we plant small symmetry-breaking parameters of size $\delta$. These include center-of-mass offsets, axis tilts, and friction asymmetry, alongside the existing mass, friction, and damping parameters. We then calibrate from $n$ synthetic noisy trajectories.

The prediction is a log-log slope of $-1/4$ for the planted parameters' error against $n$ under centered motions, and $-1/2$ under symmetry-breaking motions. We also predict a crossover at roughly $n \sim \delta^{-4}$, where the true asymmetry becomes resolvable. Regular parameters should show $-1/2$ throughout. A slope of $-1/2$ everywhere would refute the core claim.

We then apply the diagnostic to the 31-versus-57 parameter sets of Collins et al. to test whether the added parameters are disproportionately symmetry-singular or flat. On DEAL's 125-parameter AllegroHand environment, we test whether collapse concentrates in the flagged directions.

The real-world check uses physical pushing of cubes, cylinders, and cones on a Kinova-class arm with motion capture. We will use the Collins et al. trajectories if they can be obtained, and otherwise re-collect matched trajectories. The comparison is between real-data budgets for centered and designed off-center pushes. Success is measured by held-out trajectory error and by the spurious-pitch failure mode, at equal numbers of real trials.

Finally, on REALM we recalibrate the 14 parameters with our designed motions and report whether sim-real correlation on object-position stress tests holds at or above 0.921 with fewer calibration trials. We also check whether profile-likelihood and distance-prior intervals reach nominal coverage where Wald intervals, which assume the $n^{-1/2}$ rate, under-cover.