Robot arms that learn from demonstrations usually output where the hand should go next, and a low-level controller then tracks that target as rigidly as it can. That works in free space and fails in contact. When a plug is slightly misaligned, a stiff arm pushing toward a slightly wrong target wedges and jams. We propose a simple fix. A diffusion policy is a generator that produces an action by gradually denoising random noise, and if it is asked several times what to do, its answers scatter more along some directions than others. In directions where human demonstrators were inconsistent, the policy's answers scatter widely, and the arm should yield there. In directions where demonstrators were consistent, the answers cluster tightly, and the arm should hold firm. We turn that scatter, computed fresh for every action chunk at no extra training cost, into a direction-dependent stiffness. A fast inner controller then renders that stiffness on a quasi-direct-drive (QDD) arm, meaning an arm driven by strong motors through very little gearing, so it can be pushed back by hand and can render low stiffness faithfully.
The opportunity comes from putting three facts side by side. Diffusion policies capture several distinct ways of doing a task without blending them into an invalid average (Diffusion Policy). Splitting control into a slow learned planner and a fast compliant loop is what makes contact tasks work: in FILIC, a 25 Hz policy paired with a 2 kHz impedance loop reached 80% real plug insertion, against 46.7% for position-only control. QDD hardware such as Blue and IRIS is cheap and naturally compliant. What has not been tried is using the distribution the generative policy already represents as the signal that tells the fast loop how compliant to be. Existing systems instead either learn stiffness as a separate supervised output or set it by hand.
Core Claim
The claim is that, within a single behavioral mode, the spread of diffusion-policy samples over the next equilibrium positions is a usable estimate of which task-space directions tolerate error. Converting it to stiffness inversely should give contact performance that matches or beats a separately learned stiffness and beats the best hand-tuned fixed stiffness, without any stiffness labels. Spread across modes means something different. It signals ambiguity about which strategy to follow, not tolerance, so it must be separated out: the controller commits to one mode for the whole chunk and computes spread only among samples from that mode. The claim is falsifiable at two levels. First, the measured spread should align with the true contact-constraint directions; for an insertion, lateral spread should be large and axial spread small near the hole. Second, scheduling stiffness from the spread should reduce peak contact force and jamming relative to fixed-gain control at equal success.
Concretely, at each replanning step we draw $N$ chunks in parallel from different noise seeds. We cluster them, keep the cluster containing the sample we will execute, and compute $\Sigma_t$, the $3\times3$ covariance of that cluster's end-effector equilibrium positions at chunk step $t$. The commanded translational stiffness is then
$$K_t = k_{\max}\left(I + \Sigma_t/\sigma_0^2\right)^{-1},$$
where $k_{\max}$ is the stiffest gain the hardware renders stably, $\sigma_0$ is a length scale below which spread is treated as noise, and $I$ is the identity matrix. Damping is set for critical damping from $K_t$ and the arm's task-space inertia, and $K_t$ is clipped from below at a floor $k_{\min}$. This is the only equation the method needs. Rotational stiffness is handled the same way using the tangent-space covariance of orientations. Directions with spread much smaller than $\sigma_0$ get stiffness near $k_{\max}$, and directions with spread much larger get proportionally softer.
Relation to Prior Work
Two lines of work learn compliance and supply our closest competitors. DBIL trains a transformer diffusion model, conditioned on measured wrenches, to reconstruct a zero-force equilibrium trajectory. It then adapts directional stiffness reactively from sensed resistance, reaching 100% zero-shot insertion of cylindrical, square and star pegs on a geared arm. DCM uses a diffusion model offline as a stand-in contact simulator to tune open-loop stiffness profiles with 75% fewer physical trials. Both treat stiffness as something to fit or react with. We instead read stiffness predictively from the policy's own sample spread before contact forces arise, and we need neither force-conditioned training nor offline optimization. DBIL is therefore a baseline, not a method we reuse.
FILIC supplies our control architecture: a slow pose-predicting outer loop and a fast impedance-torque inner loop. FILIC's outer policy is deterministic regression with fixed gains. Its 46.7%-versus-80% result motivates compliance in general but does not tell us whether stiffness should vary. IRIS supplies the hardware recipe: Unitree GO-M8010-6 QDD motors, a belt-driven elbow, a 10 Hz chunking policy over a 200 Hz joint impedance loop. IRIS's tasks are free-space camera motion, and it uses a conditional variational autoencoder (CVAE), not diffusion.
Hierarchical systems like HiRT and SuSIE decouple slow reasoning from fast execution through latents or image subgoals. SuSIE's authors note that this interface carries no information about stiffness or contact. Our interface is narrower in scope, covering only the outer policy and the inner loop, but richer in physical content: each chunk ships equilibrium poses plus a stiffness matrix derived from the policy's own uncertainty. We do not add a vision-language planner. Doing so would test something else.
Two papers constrain the design. Demystifying Action Space Design found, across 13,000+ rollouts, that chunk-relative deltas beat step-wise deltas by over 10% and that diffusion handles joint space well. Stiffness, however, is a task-space quantity: Cartesian impedance in FILIC and DBIL is defined over end-effector directions. We therefore compute spread in task space via forward kinematics even if the policy predicts joints. Blue's 2.6 Nm static hysteresis on a 2-DoF differential sets a floor below which low stiffness cannot be rendered faithfully, which is why $k_{\min}$ exists and why friction compensation in the inner loop is part of the build.
Research Program
The hardware is an IRIS-style 6-DoF arm on GO-M8010-6 QDD actuators, with a compliant parallel gripper. The inner loop runs Cartesian impedance at 500 Hz to 1 kHz with gravity and friction feedforward. External wrenches are estimated from motor currents through a FILIC-style digital-twin comparison. An external force/torque sensor is mounted only for evaluation and is never used by any policy.
Demonstrations are collected by kinesthetic teaching, which QDD backdrivability makes easy. Fifty to one hundred episodes are collected per task, matching the data regime of Diffusion Policy. We deliberately do not instruct demonstrators to be consistent, because their natural variability is the signal.
The outer policy is a standard convolutional Diffusion Policy with chunk-wise delta actions, predicting 16 steps and executing 8. It is trained on pose only. We draw $N=16$ samples per replan in one batched pass, so latency barely rises over a single sample on a GPU. Mode clustering uses agglomerative clustering on chunk endpoints with a threshold chosen on held-out demonstrations.
The analysis work has two parts. The first measures how well $\Sigma_t$ aligns with constraint directions. We record the angle between the top eigenvector of $\Sigma_t$ and the hole's lateral plane, as a function of distance to contact. The second checks calibration. Diffusion samplers may under- or over-disperse, so we compare $\Sigma_t$ to the empirical spread of demonstrations at matched states found by nearest-neighbor lookup.
Baselines
The naive reference point is position control: the same diffusion policy with the inner loop set to maximal stiffness everywhere, which is the regime where FILIC saw real plug insertion drop to 46.7%. Run with identical demonstrations and policy weights, it establishes how much any compliance helps on QDD hardware.
The strongest simple competitor is the best fixed Cartesian stiffness. We sweep a grid of diagonal gains (isotropic values plus axis-aligned anisotropic settings that a sensible engineer would pick for insertion) with the same policy weights, and give it a tuning budget of physical trials equal to what we spend choosing $\sigma_0$ and $k_{\min}$. Beating it means higher success or equal success with significantly lower peak contact force on the evaluation sensor.
The strongest existing method for adaptive compliance is DBIL's reactive directional stiffness adaptation. We implement its rule of lowering stiffness along directions of measured resistance using our current-based wrench estimate, on top of the same diffusion pose policy. DBIL reports 100% zero-shot peg insertion on its own geared platform. That number is not transferable to our arm, so we re-run it on identical hardware and tasks.
The deterministic-architecture competitor is FILIC itself: a transformer regression outer loop with force input and fixed impedance. FILIC reports 80% real power-plug insertion. We re-implement it on our arm with the same demonstrations, giving us a direct estimate of what the generative policy plus spread-derived stiffness adds over a unimodal policy.
Three ablations isolate the key ingredient. The first computes covariance across all samples rather than within the committed mode. We predict this softens the arm wrongly at decision points such as choosing which side to push a T-block. The second keeps the time-averaged stiffness magnitude but randomizes its direction, testing whether direction rather than overall softness matters. The third substitutes a regression policy with a learned per-state variance head, testing whether diffusion's sample spread carries information a Gaussian variance does not.
Every learned component gets the same demonstrations, the same training compute, and the same number of tuning trials. Evaluation is twenty or more trials per condition with randomized object poses.
Why not just…?
Why not just run the diffusion policy with a fixed low stiffness, since QDD arms are already compliant? This is the most serious threat. A soft arm may insert plugs simply by sagging into place. Low uniform stiffness, however, costs tracking accuracy in the directions that matter, such as axial insertion depth and pushing a heavy object. Blue's hysteresis also makes very soft settings imprecise. The fixed-gain sweep settles this directly. If the best fixed gain matches spread-scheduled stiffness on both success and peak force across all tasks, the core claim fails and we will report that.
Why not just learn stiffness as an extra output, as variable-impedance work does? That requires stiffness labels, which kinesthetic or teleoperated demonstrations do not provide. DCM needed offline Bayesian optimization over a learned contact model to obtain stiffness profiles, and DBIL needed wrench-conditioned training on 1.6 hours of data. Our method uses no labels. If the regression-with-variance ablation performs as well, the lesson would be that any calibrated uncertainty suffices, which would be a useful negative result about the need for diffusion specifically.
Why not just react to sensed force, lowering stiffness once contact pushes back? Reaction happens after the jamming force has begun, and current-based force estimates on QDD motors are corrupted near hysteresis deadbands. Predictive softening acts before contact. The DBIL-style baseline tests exactly this. We expect the gap to be largest in peak force at first contact, not in final success, and we will measure both. A combination of predictive and reactive control is a natural follow-up if both help.
Proposed Experiments
We use four tasks chosen from the benchmarks in the file. Real power-plug insertion follows the FILIC protocol and tests a narrow constraint direction. Peg insertion with cylindrical, square and star pegs follows DBIL and tests whether the spread rotates with geometry. Real Push-T follows Diffusion Policy, where Diffusion Policy reports 95% success against 20% for the best baseline; it is used to test multimodal decision points where cross-mode spread must not be mistaken for tolerance. One FurnitureBench insertion subtask is included because full assembly there completes only about 2 of 12 subtasks for behavior cloning, so we treat it as a stress test, not a headline.
Metrics are success rate, peak and integrated contact force measured by the evaluation-only sensor, jamming events (stalls exceeding a force threshold for over 0.5 s), and completion time.
The decisive checks are three. First, the alignment between the dominant spread direction and the true constraint direction should exceed chance well before contact. If it does not, the mechanism is not doing what we claim, regardless of success rates. Second, on plug and peg insertion, spread-scheduled stiffness should reduce peak contact force relative to the best fixed gain at equal or higher success. Third, on Push-T, within-mode covariance should beat all-sample covariance. This would confirm that separating ambiguity from tolerance is necessary and that the multimodal policy class is doing real work, not just a smoothing generator.
A final cross-hardware check repeats plug insertion with added inner-loop friction to emulate a high-ratio drive. We expect the advantage of low-stiffness directions to shrink, which would show that the method's value depends on the transparency QDD hardware provides.
References
- Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
- FILIC: Dual-Loop Force-Guided Imitation Learning with Impedance Torque Control for Contact-Rich Manipulation Tasks
- Quasi-Direct Drive for Low-Cost Compliant Robotic Manipulation
- IRIS: Learning-Driven Task-Specific Cinema Robot Arm for Visuomotor Motion Control
- Diffusion-Based Impedance Learning for Contact-Rich Manipulation Tasks
- A Contact Model based on Denoising Diffusion to Learn Variable Impedance Control for Contact-rich Manipulation
- HiRT: Enhancing Robotic Control with Hierarchical Robot Transformers
- Zero-Shot Robotic Manipulation with Pre-Trained Image-Editing Diffusion Models
- Demystifying Action Space Design for Robotic Manipulation Policies