Transferring Speed–Energy Trade-offs from Simulation to Cheap Robot Arms

A controller for a $2,000 robot arm usually has to balance competing goals. A fast move costs more motor current and heat, and a smooth move is slower. A preference-conditioned policy is one network that takes a dial setting for "how much do I care about speed versus energy versus accuracy" and returns the matching behavior. In simulation such a policy can trace out the whole menu of best-possible compromises, called the Pareto front. On a low-cost arm, however, the simulator is wrong in a structured way. Gear backlash matters more at high speed, plastic links bend more under high acceleration, and servos lose torque as they heat. These effects interact, so the simulator's errors are not the same everywhere on the menu. One dial setting may transfer almost perfectly while another lands somewhere unexpected. Worse, two policies may swap order, so the one that looked strictly better in simulation turns out strictly worse on hardware.

This note proposes to measure that distortion directly and then correct it. The method spends a small, statistically designed batch of real-robot runs, chosen in the style of classical design-of-experiments, to find which pairs of physical parameters jointly drive the sim-to-real error in the objectives. It then fits a correlated randomization distribution over only those pairs and retrains the preference-conditioned policy under it. The core claim is testable and could be false. On accessible arms, sim-to-real error reorders the Pareto front, rather than merely stretching it, and the reordering is explained by a few two-way parameter interactions. If the error only stretches the front, a cheap relabeling fix suffices, and the proposal says up front that it would then pivot to that.

Core Claim and What Is New

Our hypothesis concerns policies trained under standard independent domain randomization, meaning each simulator parameter is perturbed separately within a fixed range. When such a policy is deployed on a low-cost arm, a measurable fraction of the dominance relations among its preference settings will flip, concentrated at the fast end of the front. A randomization distribution fitted to the few interaction terms that a real-world screening experiment identifies will reduce those flips and raise real hypervolume more than either independent randomization or a fitted distribution with the same marginals but no correlations, at equal real-robot budget. Hypervolume here is the volume of objective space dominated by the achieved front.

The closest work is Monolithic vs. hybrid controller for multi-objective Sim-to-Real learning. It transfers a reach-and-avoid task from CoppeliaSim to a Franka Panda and shows that two decoupled policies with a switching threshold beat one scalarized DDPG policy on hardware. That work motivates this proposal, since it shows that multi-objective structure survives transfer badly when objectives are blended. It differs in three ways. The robot is a rigid arm costing more than $30k, so backlash and compliance barely matter. The trade-off is a single hand-set threshold rather than a continuous front. The paper also never asks which physical parameters cause the transfer error.

Our policy representation comes from Learning Pareto Set for Multi-Objective Continuous Robot Control. Its hypernetwork maps a preference vector to policy weights and represents a whole continuous front compactly: about 140k parameters versus 29M for PG-MORL's archive on MO-Hopper-v3. Its evaluation is purely in MuJoCo. The authors also note that linear scalarization misses concave front regions, which constrains how we read hypervolume results.

The sim-to-real side starts from domain randomization (Simulation-based reinforcement learning for real-world autonomous driving) and its adaptive variants. Active Domain Randomization learns where to sample physics parameters and gave tighter outcomes than uniform randomization on the low-cost Poppy Ergo Jr. arm. Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design generates environments by maximizing regret between two agents. Both optimize a single scalar reward and treat parameters as separate sampling dimensions. The posterior-inference line, Neural Posterior Domain Randomization and BayesSimIG, infers simulator parameters from real data and can in principle capture correlations. Its target is matching trajectories of one behavior, not matching objective vectors across a spread of trade-offs. We treat that line as the strongest competitor rather than as solved prior art.

From the manufacturing literature we borrow the methodological stance, not the domain. Effect of some process parameters on geometric errors in twopoint incremental forming shows the effect of feed rate reversing with tool diameter and step size. Pelletization of Sunflower Seed Husks … by Response Surface Methodology shows a moisture effect that inverts past a threshold depending on pressure. Both show that tuning one factor at a time strands you on the wrong optimum. Equally relevant is a contrary result. Application of Design of Experiment Method for Thrust Force Minimization in Step-feed Micro Drilling found its interaction terms statistically insignificant, with feedrate dominating at F = 65.3. Interactions are not guaranteed, so our first experiment checks whether they exist on low-cost arms at all.

Research Program

The testbed is a REPLAB-style cell (REPLAB): a WidowX-class arm, an RGB-D camera, and REPLAB's calibration protocol, with at least two nominally identical units. The task is the reach-and-avoid setup of Dag et al., ported to this arm. The arm reaches a tracked target while avoiding an obstacle. We score three objectives: transit time, electrical energy integrated from servo current readings, and final end-effector error. The third objective is where backlash shows up.

In the first stage, a simulator model of the arm exposes backlash, joint friction, link stiffness, servo torque limit, a temperature-dependent torque derating, and control latency. A hypernetwork policy is trained under independent randomization of all of these.

In the second stage, a screening experiment runs on the real arm. We choose a small set of preference settings that span the simulated front. Each is run under a two-level fractional factorial over conditions we can physically set: payload mass, commanded speed cap, control rate, and motor warm-up state, the last obtained by running a duty cycle beforehand. Each run yields a measured objective vector, and the identical condition is simulated at nominal parameters. Analysis of variance on the gap between real and simulated objectives identifies which main effects and two-way interactions are significant. An example would be speed cap × warm-up, if hot servos are what make fast policies overshoot.

In the third stage, simulation-based inference fits a joint distribution over the simulator parameters implicated by those terms. Its likelihood is the match to the measured objective vectors across all preference settings. The policy is then retrained under this correlated distribution. Restricting the fit to the screened terms keeps the inference low-dimensional, which is the point of screening first. The total real-robot budget is fixed in advance at a few hundred episodes, well under one REPLAB-style training day.

Baselines

The naive reference is the hypernetwork policy trained under independent uniform randomization and deployed zero-shot. It shows how large the problem is. If its dominance-flip rate is near zero, the proposal's premise fails.

The scalarized reference follows Dag et al.: one DDPG policy per preference weight, trained on a weighted-sum reward, plus their hybrid switching controller with the threshold swept across settings. Both use the same simulator, randomization, and total environment steps. Dag et al. report 90% success and 10% collisions for the hybrid controller on a Franka, versus 30% success and 70% crashes for the monolithic one. That is the only multi-objective sim-to-real number in the file. We expect the hybrid to fall below its Franka number on a backlash-prone arm, and we beat it only if our policy matches its success rate while dominating it in time and energy.

The strongest learned competitor is posterior-fitted randomization in the spirit of Neural Posterior Domain Randomization and BayesSimIG, fitted on all simulator parameters with the same real episodes. I found no reported result on Lacuna for these methods on low-cost arms or on multi-objective metrics, so we run them ourselves with the same inference network and simulation budget.

Adaptive randomization (Active Domain Randomization) is run with its sampler driving the hypernetwork's training environments. For it, the file reports only qualitative tighter spread on the Poppy arm and no hypervolume number.

The key ablation keeps our pipeline but replaces the fitted joint distribution with its product of marginals. This isolates whether correlation structure, rather than better-centered ranges, does the work.

Beating a baseline means significantly lower dominance-flip rate and higher real hypervolume, with paired runs and a bootstrap over episodes, at equal real-episode count. It must hold on the second, unseen unit as well as the screened one.

Why not just…?

Why not just relabel the simulated front with a correction learned from real runs? Suppose sim-to-real error only rescales the objectives monotonically. Then every policy keeps its rank, and a regression from simulated to real objective vectors, fitted on the same screening runs, recovers the real front at almost no cost. This is the most serious threat. The first experiment, measuring the dominance-flip rate under independent randomization, settles it directly. If flips are rare, we adopt relabeling and report that interaction-aware retraining is unnecessary on this hardware class. We include relabeling as a baseline throughout.

Why not just identify the simulator once and retrain? A point-estimate system identification is cheap. It ignores that the gap changes with operating state. Warm-up and payload shift effective torque and compliance, which is exactly the non-stationarity flagged as unaddressed in classical interaction studies. It also does nothing for inter-unit variability. The cross-unit test and the warm-up factor in the screening design expose this. A point estimate can be included as a degenerate posterior.

Why not just optimize the trade-off on hardware with Bayesian optimization? If the policy reduced to a few trajectory knobs, direct hardware search could beat any simulator. Our policy is closed-loop and must produce a continuous front over obstacle placements, so hardware search must repeat per setting. REPLAB needed about 24 hours of real training for single-objective reaching, and low-cost servos wear under that use. We still run hardware Bayesian optimization over a parameterized trajectory family at equal episode budget. If it matches our hypervolume, that is reported as a negative result for simulation on this task.

Why not just collect demonstrations, as ALOHA does? Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware shows that 50 demonstrations give 80–90% success on fine tasks with cheap arms. Human demonstrations, though, sample one implicit trade-off and give no systematic coverage of the energy–speed front. Imitation answers a different question from ours.

Decisive Checks

Three results decide the proposal. The first is the dominance-flip rate of the independently randomized policy on real hardware, broken down by preference region. Without non-trivial flips the main claim is dead and relabeling wins. The second is the screening analysis of variance. If no two-way interaction is significant, as in the micro-drilling study, the correlated distribution should equal its marginal ablation and the interaction story is refuted. The third is the head-to-head at equal budget on the held-out second unit, measuring flip rate, real hypervolume, success and collision rates, and servo faults per hundred episodes. Our method should beat both posterior-fitted randomization over all parameters and the marginal ablation. Gains confined to the screened unit would indicate overfitting to one robot rather than capturing hardware-class interactions.