Robots trained in simulation are usually made robust by domain randomization: each training episode draws a fresh set of physics parameters, such as friction or mass, so the policy learns to cope with whatever the real world turns out to be. In practice those parameters are a handful of scalars. Real objects do not vary that way. A box is slippery on one face and tacky on another, its mass sits off-center, a table has a wet patch. These are fields, values that vary over a surface or volume, and once they are discretized they become hundreds of parameters. In a space that large, uniform sampling almost never hits the narrow set of configurations where the policy fails. Methods that actively search for hard training tasks were built for fewer than about ten scalars.
We propose a task sampler for field-valued randomization. Its core is a neural operator, a network that maps one function to another and can be evaluated at any grid resolution, together with cheap uncertainty estimates. The sampler does not predict a single "how badly did this episode go" number. It predicts the spatial and temporal outcome of a rollout: where on the object contact slipped, and how the object's pose error evolved over time. The risk score is then computed from that prediction by a fixed, known formula. The bet is that each expensive rollout gives the sampler far more supervision when it learns from these dense outcome fields than when it learns from one scalar. That extra supervision is what lets surrogate-guided task selection keep working as the randomized space grows from tens to thousands of dimensions.
Core Claim
Surrogate-guided task sampling for robust reinforcement learning is limited mainly by how little each rollout tells the surrogate, not by the choice of acquisition rule. With field-valued randomization, a surrogate that predicts dense outcome fields and derives risk from them should find failure-inducing training environments with fewer rollouts than a surrogate that regresses scalar risk directly. The gap should widen as field resolution grows. This holds even when both surrogates share the same input encoding, the same posterior-sampling rule and the same diversity penalty. The prediction is falsifiable: if the dense-output advantage does not grow with dimensionality, the claim fails.
Where This Sits
The closest prior work is Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments (PDTS). PDTS trains a surrogate that predicts the risk of each candidate task. It selects tasks by drawing one sample from that surrogate's posterior, and it adds a Euclidean diversity penalty to avoid a proven "concentration curse", in which picking the hardest tasks collapses training onto an unrepresentative cluster. PDTS supplies our selection mechanism and our main baseline. It differs from this proposal in two ways: its tasks are low-dimensional scalar vectors, and its surrogate sees only a scalar outcome.
Two other active methods motivate the scalability gap. Active Domain Randomization drives a population of parameter proposals with an adversarial discriminator; it was tested up to 8 dimensions and has no explicit model of failure. Data-Efficient Domain Randomization With Bayesian Optimization fits a Gaussian process over randomization hyperparameters. It needs real-robot returns in the loop, and its authors note that the Gaussian process does not scale to high-dimensional distributions. Two further methods, Neural Posterior Domain Randomization and Sim-to-Real Transfer with Neural-Augmented Robot Simulation, fit the simulator to real data. They are outside the zero-shot setting we target, and we do not compete with them.
The surrogate side supplies the method. Composite Bayesian Optimization in function spaces using NEON—Neural Epistemic Operator Networks attaches a small uncertainty head to an operator backbone. The head is conditioned on a random index vector, so each sampled index gives a plausible model, and it uses 10 to 100 times fewer parameters than ensembles. NEON optimizes a known function of the operator's predicted output field, and it is the source of our "dense output, fixed risk formula" structure. Thompson Sampling in Function Spaces via Neural Operators shows that a trained neural operator can act as an approximate posterior sample. It reaches a final regret of 0.012 on Darcy flow, against 0.703 for standard Bayesian optimization, with outputs of 6,144 dimensions where Gaussian processes became intractable.
Neither line of work has been pointed at a policy's failure modes. Their targets are fixed physical systems; ours is a policy that changes as training proceeds. That non-stationarity is the main new technical difficulty.
On the robotics side, Rapid Motor Adaptation for Robotic Manipulator Arms shows that naive state-based randomization can collapse completely: 0.3% success on YCB pick-and-place, against 73.8% for its adaptive policy. Its randomization itself is uniform. Intersection-free Robot Manipulation with Soft-Rigid Coupled Incremental Potential Contact shows that better contact physics alone yields 70% zero-shot real success. It cannot, however, tell a policy which friction or stiffness the real object will have, so it constrains rather than replaces our approach.
Research Program
Environments with field-valued parameters. We start from ManiSkill2 tasks (ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills): PickCube, YCB pick-and-place and 3 mm peg insertion. Rigid-body engines assign one material per collision shape, so we approximate spatial fields by splitting each object surface and the table top into patches, each with its own friction coefficient. Each object's volume gets a mass-density field, which shifts its center of mass and inertia.
Fields are drawn from Gaussian random fields with controllable smoothness. We vary resolution from 4 to 1,024 patches per surface. That sweep is the central experimental axis.
Surrogate. For each candidate task, the surrogate takes the input fields on their native grid together with a short summary of the current policy. It predicts two output fields:
- an accumulated contact-slip map over the object's surface patches;
- a time series of object pose error.
These targets come for free from the simulator on every rollout. The risk score is a fixed formula applied to the predicted outputs: a smoothed indicator that terminal pose error exceeds the task tolerance. Averaging that score over the worst decile of candidates gives the conditional value at risk (CVaR$_{0.9}$), the mean outcome in the worst 10% of cases.
The backbone is a Fourier neural operator with a NEON-style uncertainty head. Each training batch is selected PDTS-style. We draw one random index, which fixes one plausible surrogate, score a candidate pool 64 times the batch size, and pick a high-risk subset while penalizing similarity between input fields. Similarity is measured with a Sobolev norm on the fields, which compares both values and smoothness and gives the same answer regardless of patch resolution.
We do not claim this diversity measure as a contribution. On a truncated spectral basis it reduces to a weighted Euclidean distance, and the baselines below exploit exactly that.
Handling a moving target. The policy improves during training, so old rollouts become stale. We will refit the uncertainty head on a recency-weighted buffer every iteration and the backbone less often, using NEON's stop-gradient split between the two. We will measure how the surrogate's rank correlation with true risk decays with data age.
Transfer. Whichever sampler wins in simulation is used to train both a plain policy and an RMA²-style adaptive policy. We evaluate on held-out field families that never appear in training: piecewise-constant "contaminated patch" fields and fields at resolutions the surrogate never saw. If hardware is available, we add zero-shot trials on a Franka arm with objects deliberately given non-uniform surfaces, for example with taped or oiled regions.
Baselines
Uniform randomization (the "do nothing" reference). Fields are sampled from the training prior with no selection. We report its mean and CVaR$_{0.9}$ success at every resolution. If our method cannot beat it at the lowest resolution, the experimental setup is broken.
PDTS on spectral coefficients (strongest existing method). Each field is projected onto its leading $K$ Karhunen–Loève modes, the principal components of the random-field prior. PDTS then runs unchanged: a scalar risk surrogate, posterior sampling, and Euclidean diversity on the coefficients. This is the most direct way a practitioner would apply existing work to our setting.
PDTS reports improvements of over 15% in CVaR$_{0.9}$ against uniform training on most of its meta-RL benchmarks, up to 73% better out-of-distribution performance on LunarLander, and up to 2.4× faster convergence on Pusher (Fast and Robust). No reported result exists on Lacuna for PDTS, or for any active sampler, on field-valued manipulation tasks.
We tune $K$ and the diversity weight for PDTS with the same search budget as our method. Beating PDTS means higher CVaR$_{0.9}$ success on held-out fields at matched total simulator steps, with the margin growing with resolution.
Brute-force evaluation. Every candidate in the pool is actually rolled out, and the hardest diverse subset is selected using true outcomes. GPU simulation makes this affordable at small pool sizes. It is the ceiling a surrogate is trying to approximate cheaply. The evaluation rollouts count toward its step budget, so the comparison is fair.
Active Domain Randomization, run on spectral coefficients, represents the discriminator-based alternative.
Scalar-output ablation. This isolates the key ingredient. It uses the identical operator backbone, uncertainty head, selection rule and diversity penalty, but is trained only on the scalar risk of each rollout. If it matches the full method, the dense-output claim is false.
All methods share the same PPO learner, simulator, step budget (selection rollouts included), seeds and hyperparameter search budget. We also report surrogate wall-clock time.
Why not just…?
Why not just run PDTS on a low-dimensional encoding of the field? This is the most serious threat. Smooth fields are well described by a few spectral modes, and on those modes Euclidean diversity matches our function-space diversity. We expect PDTS on coefficients to be competitive at low resolution and with smooth fields.
Our claim is narrower: when failures depend on fine spatial structure, for example a single slippery patch under the gripper finger, a scalar surrogate needs many more rollouts to locate that structure than a surrogate supervised with slip maps. The resolution sweep, together with the rough-field and contaminated-patch conditions, settles this directly. If PDTS keeps pace at 1,024 patches, we will report that the operator machinery is unnecessary.
Why not just simulate every candidate, since simulation is cheap? For pools of a few hundred candidates in a fast engine, this may genuinely win, and the brute-force baseline tests it at matched step budgets. A surrogate earns its place only when candidate pools are large or rollouts are expensive, for example in contact-rich soft simulation of the kind ZeMa makes possible. We will plot the crossover point against pool size rather than assume it.
Why not just train an adaptive policy that infers the field online, and keep uniform randomization? RMA²'s results show adaptation helps a great deal. Its policy, however, still learns only from the training distribution it is shown. If uniform sampling rarely produces the failure-inducing fields, the adapter rarely learns to recognize them. We test this by training the RMA²-style policy under both uniform and surrogate-guided sampling. The two approaches are complementary unless the adaptive policy shows no gain from better task selection, which would be an informative negative result.
Why not use a Gaussian process, as BayRn does? BayRn's own authors report that Gaussian processes break down in high dimensions. NOTS also shows Gaussian-process baselines failing on 6,144-dimensional outputs. We nonetheless include a Gaussian process on spectral coefficients at low $K$ so that this failure is shown in our own setting rather than assumed.
Decisive Experiments
The headline experiment plots held-out CVaR$_{0.9}$ success against field resolution (4 to 1,024 patches) for every method, at a fixed simulator-step budget, on PickCube, YCB pick-and-place and 3 mm peg insertion. Within-simulation reference points are RMA²'s 51.6% on peg insertion and ManiSkill2's 91% simulated PickCube success. The proposal succeeds if two things hold: the full method beats PDTS-on-coefficients by a margin that grows with resolution, and the scalar-output ablation tracks PDTS rather than the full method.
Two supporting checks follow:
- Information per rollout. We measure how well each surrogate ranks true risk as a function of rollouts seen, which is the mechanism behind the headline result.
- Resolution invariance. A surrogate trained at 64 patches is evaluated at 1,024 without retraining, testing the zero-shot super-resolution property reported in Scattering with Neural Operators in a control setting.
The real-robot trials, if available, compare zero-shot success on non-uniform objects for policies trained under uniform sampling, PDTS-on-coefficients, and the proposed sampler. ManiSkill2's 60% zero-shot real PickCube success under standard training is the reference point.
References
- Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments
- Active Domain Randomization
- Data-Efficient Domain Randomization With Bayesian Optimization
- Neural Posterior Domain Randomization
- Sim-to-Real Transfer with Neural-Augmented Robot Simulation
- Composite Bayesian Optimization in function spaces using NEON—Neural Epistemic Operator Networks
- Thompson Sampling in Function Spaces via Neural Operators
- Rapid Motor Adaptation for Robotic Manipulator Arms
- Intersection-Free Robot Manipulation With Soft-Rigid Coupled Incremental Potential Contact
- ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills
- Scattering with neural operators