Diffusion and flow policies are robot controllers that learn to turn random noise into an action sequence. They are trained on a loss that is an average over random noise draws and random "noise levels," and each training step estimates that average from a single draw per example. This proposal starts from a simple observation. When the robot's own gripper hides the target, as it does in the last centimeters before contact, the camera image no longer pins down the correct action. Training examples from those moments have a much noisier loss than examples from free-space reaching. Uniform sampling gives every example the same number of draws, so it spends most of its noise-sampling budget on frames where it is not needed and too little on the occluded frames that decide whether a grasp or insertion succeeds.
We propose to give each training example a number of extra noise draws proportional to its estimated loss spread. This is the classical optimal rule for dividing a sampling budget across groups, known as Neyman allocation. The extra draws are cheap because they reuse the image encoder's output: the camera network runs once and only the small action head is re-run. Because the allocation is computed from measured variance, it replaces the hand-tuned reuse ratio and timestep weights that variance-reduction methods usually need. The central empirical question is whether reducing this particular variance improves success on occluded terminal phases. Prior work has shown that large variance reductions sometimes leave the downstream metric unchanged, so we test this question directly rather than assume it.
Core Claim
The claim has three parts. First, in visuomotor diffusion and flow policies, the variance of the denoising loss across noise draws is concentrated in frames where the manipulation target is occluded. Second, an allocation rule driven by that variance, with no tuned hyperparameters, matches or beats the best fixed reuse ratio found by grid search. Third, this advantage appears in task success specifically on episodes whose terminal phase is occluded, at equal wall-clock training compute. If the third part fails while the first two hold, that is itself a useful result. It would identify occluded-frame estimator noise as one of the cases where variance is not the bottleneck, which is the boundary condition the literature currently lacks.
Where This Sits
The method comes from Variance Reduction for Expectations with Diffusion Teachers. That paper (CARV) reduces the variance of Monte Carlo gradients through a frozen diffusion model by caching the expensive upstream computation and re-drawing cheap noise. It reports a 2–3x effective compute multiplier in text-to-3D. CARV fixes its reuse ratio and sampling weights by hand, and it never touches robot policies. It also supplies the main constraint on our design: a variance reduction of up to 32x in single-step distillation left FID unchanged.
Variance Reduction for Policy-Gradient Methods via Empirical Variance Minimization reached a similar verdict in reinforcement learning. It found modest gains on Unlock-v0 despite large variance cuts, and its per-sample gradient computations cost $O(K)$ backward passes. That cost tells us not to compute per-sample gradients in the main loop.
AdaFlow shows that the conditional variance of the flow-matching target reflects how ambiguous the action is given the observation. It uses a learned variance network to choose solver step sizes at inference, reaching 1.17 average function evaluations. We reuse that variance estimate but move it to training, where it decides how many noise draws each example receives. AdaFlow also never separates occlusion-induced ambiguity from genuine multimodality, and our occlusion-stratified analysis does exactly that.
On the occlusion side, several papers motivate the regime. VFAS-Grasp must fall back to open-loop control in the final inch. ContactGuard predicts failures behind the occlusion but can only abort. DECO and Reactive Diffusion Policy add touch to cover blind phases. All of these change sensors or architecture. Our proposal changes only how the training loss is estimated, so it is compatible with each of them.
The classical variance-reduction line, Prox-SVRG, mini-batch semi-stochastic gradient descent and SPIDER, documents the tuning burden this work tries to avoid. These methods need inner-loop lengths and step sizes tied to unknown smoothness constants. Their finite-sum snapshot assumption also breaks under the per-step image augmentation that visuomotor training relies on, so we do not build on them directly.
Research Program
For each training example $i$ we run the image and proprioception encoder once. We then draw $K_i$ independent (noise level, noise) pairs through the action denoiser and average their losses. The total budget is fixed at $B = \sum_i K_i$. If example $i$ has per-draw loss-gradient standard deviation $\sigma_i$, the minimum-variance allocation is
$$K_i = B\,\frac{\sigma_i}{\sum_j \sigma_j},$$
with rounding and a floor of one draw per example. This follows from minimizing $\sum_i \sigma_i^2 / K_i$ subject to the budget constraint. The rule has no free parameter beyond $B$, and $B$ is set by matching wall-clock cost to the baseline.
Computing $\sigma_i$ exactly would require per-sample gradients, which is the cost that limited the empirical-variance-minimization methods. We therefore use two cheap proxies. The first is the spread of the scalar denoising loss across a small number of probe draws. The second, for flow policies, is AdaFlow's predicted conditional variance. On a subset of batches we will compute exact per-sample gradient variance, check that both proxies rank examples correctly, and report the rank correlation.
The analysis has two steps. In simulation, we render segmentation masks and compute, for every frame, the fraction of the target object and contact region visible in each camera. We then measure how the denoising-loss variance depends on this visibility fraction. This step confirms or refutes the first part of the claim before any method comparison. We also report the sensitivity surface of fixed-K training across reuse ratios and timestep weightings, at several occlusion levels. This quantifies the tuning burden in the manipulation setting and shows whether the best fixed K moves as occlusion increases.
Baselines
The naive reference is a standard Diffusion Policy (and a flow-matching policy) trained with one noise draw per example and uniform noise levels. It uses the same encoder, data, augmentation, and wall-clock budget as every other run.
The second baseline is CARV-style amortized re-noising with a fixed reuse ratio K and CARV's stratified noise-level sampling. We grid-search K and the stratum count with a tuning budget of eight full runs, and we give our method zero tuning runs. This is the strongest competitor on the actual question. If fixed K, once tuned, matches us, then our contribution reduces to removing the tuning cost. The sensitivity surface will show how large that cost is.
The third baseline is large-batch training. We drop re-noising and instead raise the batch size to consume the same wall-clock compute, with the learning rate tuned on the same eight-run budget.
The fourth baseline is AdaFlow itself. Its reported LIBERO average success is 91%, against 88% for Diffusion Policy at 20 function evaluations (AdaFlow). This is the strongest reported number in the file on a benchmark we will run. Beating it means exceeding AdaFlow's success under our reproduction, at matched training compute, and doing so mainly on the occluded-terminal subset. For RLBench we cite RVT's 62.9% average success (RVT) as context only, because RVT changes the input representation rather than the estimator.
Two ablations isolate the key ingredient. The first allocates draws by the ground-truth visibility label instead of measured variance. This asks whether occlusion itself or variance more broadly is what matters. The second allocates the same draw counts but shuffles them across examples, which removes any benefit that comes merely from having more draws in total.
Why not just…?
Why not just increase the batch size? A larger batch reduces all variance sources uniformly and pays for a full encoder pass on every additional example. Our extra draws reuse the cached encoding, so per unit of compute they target the noise-draw variance that the analysis shows is concentrated in occluded frames. The matched-compute large-batch baseline settles the comparison directly. If it ties, we will report that the reduction in variance per unit of compute did not translate into success.
Why not just tune a fixed reuse ratio well? This is the real threat. A well-tuned fixed K may recover most of the gain. Our answer is empirical. We will report how much the best fixed K shifts between low- and high-occlusion settings and how much success is lost at a mistuned K. If the surface turns out flat, the allocation rule has little to offer and we will say so.
Why not just upweight occluded frames in the loss? Reweighting changes the objective. It pushes the policy toward occluded-frame targets that are inherently ambiguous, and this risks averaging across modes. Extra draws leave the objective unchanged and only lower the variance of its estimate. The visibility-label ablation, run both as reweighting and as draw allocation, separates the two effects.
Why not just add cameras or re-render views as RVT does? Multi-view re-rendering reduces occlusion during approach. It cannot see between closed fingers at contact (RVT also depends on calibration). Our method is orthogonal to camera layout, so we will also apply it on top of a multi-view input to test whether the gain survives once occlusion is reduced.
Proposed Experiments
The main benchmarks are RoboMimic and LIBERO, since AdaFlow reports numbers on both, and a subset of RLBench tasks with fine terminal alignment. Each runs in two camera configurations, wrist-only and wrist plus a fixed view, because the wrist-only setting maximizes terminal occlusion. Success rates are reported overall and stratified by the terminal visibility fraction computed from rendered masks. The occluded-terminal stratum is the primary metric. We use at least five seeds and report confidence intervals.
For real data we use the DECO-50 dataset. It contains hand-occluded contact phases, and DECO reports 72.25% for its vision-only policy and 51.25% for Diffusion Policy. Without the robot we can only report held-out action error, split by contact and non-contact segments. We will not claim success rates there.
The decisive checks are three. The first is whether loss variance rises monotonically as target visibility falls. If it does not, the premise fails and we stop. The second is whether variance-driven allocation lowers the measured gradient variance on occluded frames at equal compute relative to tuned fixed K. The third is whether success on the occluded-terminal stratum rises while success on fully visible episodes stays flat. The second check passing while the third fails would reproduce CARV's FID result in robotics. That would be a clean negative result showing that estimator noise is not what limits occluded manipulation, and it would point effort back toward sensing.
References
- Variance Reduction for Expectations with Diffusion Teachers
- Variance Reduction for Policy-Gradient Methods via Empirical Variance Minimization
- AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policies
- VFAS-Grasp: Closed Loop Grasping with Visual Feedback and Adaptive Sampling
- ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
- DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter
- Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation
- A Proximal Stochastic Gradient Method with Progressive Variance Reduction
- Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting
- SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path-Integrated Differential Estimator
- RVT: Robotic View Transformer for 3D Object Manipulation