Choosing Action-Chunk Length by Safety Margin for Imitation Policies Under Sim-to-Real Shift

Modern robot imitation policies predict a short sequence of future actions, an "action chunk," and execute it without re-checking sensors. This open-loop commitment is a major reason these policies work: it stops small prediction errors from compounding into large drift. The same commitment conflicts with safety. Safety shields, which are monitors that override the policy just before it enters a dangerous state, need to intervene at every control step. A robot that has committed to a 16-step chunk cannot react halfway through without abandoning the chunk, and abandoning chunks is what brings compounding error back. The conflict is sharpest after sim-to-real transfer. There, unmodeled friction, latency, or contact behavior can make a chunk that was safe in simulation unsafe on hardware.

We propose to make chunk length itself the safety control. Before committing, a safety critic trained in simulation predicts the worst safety margin along each prefix of the proposed chunk. It does this across an ensemble of plausible dynamics models. The robot then executes only the longest prefix whose predicted margin stays above a threshold with high confidence. Far from danger, the robot gets long chunks and their stability benefit. Near a constraint, or where the dynamics ensemble disagrees because the real system differs from simulation, chunks shrink toward step-by-step closed-loop control. The proposal also adds the safety costs and shifted-dynamics test conditions that existing manipulation benchmarks lack, so the trade-off can be measured on standard tasks.

Core Claim

The tension is real and unresolved in the literature. Two papers show that open-loop chunking helps. Action Chunking and Data Augmentation Yield Exponential Improvements in Behavior Cloning for Continuous Spaces proves that chunked execution turns exponential error growth into horizon-free bounds, provided the system is open-loop stable. Demystifying Action Space Design for Robotic Manipulation Policies shows on real arms that chunk-wise delta actions bound error to $O(1)$ while step-wise delta actions grow as $O(k)$. In the other camp, Sim-to-Lab-to-Real uses a reachability-based critic and backup policy that check and override every step. Neither line says how to combine the two.

Our claim is that a single, commit-time choice of chunk length, made from predicted safety margin and dynamics uncertainty, beats both fixed-length chunking and per-step shielding on the joint metric of task success and constraint violations under held-out dynamics shift. The chunk length is

$$k_t = \max\{\,k \le K : \hat{P}_t\big(\min_{j \le k} h(s_{t+j}) \ge 0\big) \ge 1-\delta\,\}.$$

Here $K$ is the policy's maximum chunk length, $h$ is a signed safety margin (positive inside the safe set), $s_{t+j}$ are states reached by executing the proposed chunk under each member of a dynamics ensemble, $\hat{P}_t$ is the fraction of ensemble members predicting the prefix stays safe, and $\delta$ is the risk tolerance. When even $k=1$ fails the test, a backup policy takes over, as in shielded reinforcement learning.

Research Program

We will build three components and one benchmark extension.

Chunk-level safety critic. We train a critic that maps a state and a candidate chunk prefix to the minimum future safety margin. It uses the discounted safety Bellman formulation from Sim-to-Lab-to-Real, which we reuse as the method. The difference is that we evaluate it over action sequences rather than single actions.

Avoiding the safety paradox. Breaking Safety Paradox with Feasible Dual Policy Iteration identifies a known failure: a safe learner stops visiting the safety boundary, so its critic loses the data it needs there. We avoid this by generating near-boundary training data in simulation, where violations are free. We use disturbances that push rollouts toward failure, in the spirit of SAFE-GIL, which motivates the design. SAFE-GIL computed such disturbances by exact reachability in low dimensions. We use learned critics instead and do not require exact reachability.

Ensemble over randomized dynamics. Each critic and dynamics member is trained on a different slice of the domain-randomization range. Their disagreement then serves as a sim-to-real uncertainty signal. This plays the role that the mismatch region plays in Provably Safe Sim-to-Real Transfer, which proves zero unsafe exploration only for tabular problems under a separability assumption. We give up that guarantee in exchange for continuous, high-dimensional control, and we measure violations empirically rather than certifying them.

Rejected chunks as labelled data. Chunks rejected on the shifted system are logged as states where the policy needed short horizons. These states form a small, targeted set for later supervisor relabeling.

Benchmark extension. We add constraint costs to existing benchmarks: contact-force thresholds, joint-torque limits, and workspace bounds. They go on robosuite peg insertion and nut assembly, and on Franka tasks in RoboHive. RoboHive's sim/hardware toggle gives a path to physical evaluation. For each task we define "shifted" dynamics with friction, mass, actuation latency, and controller gains outside the training randomization range. These play the role of a surrogate real world, following the modified-simulator evaluation in RGAT, and are followed by a small physical Franka study. The extension supplies what Safety Gymnasium leaves open: its own authors note that safe transfer to hardware is unaddressed, and its suite contains no imitation setting. We also include Safety-Gymnasium Point and Car Goal tasks, with demonstrations generated by SafePO policies, as a navigation check where per-step shielding is known to work.

Baselines

The naive reference is plain behavioral cloning with the same policy backbone (a chunked transformer policy, chunk-wise delta actions) executed with a fixed chunk of length $K$ and no safety mechanism. On the shifted dynamics this shows how much the reality gap alone costs. For context, SAFE-GIL reports roughly 80% failure for plain behavioral cloning with 8 demonstrations in simulated ground navigation, against about 10% for its method. That gap is the size of effect a safety-aware data scheme can produce, although on a different task.

The strongest existing competitor for the imitation side is ThriftyDAgger. It gates control on ensemble novelty and a learned task-failure critic, and on robosuite peg insertion reports a 73% autonomous success rate, against 48% for LazyDAgger and 24% for SafeDAgger. We run it on the same demonstrations, with the same supervisor budget for the rejected-chunk relabeling stage and the same backbone. Its risk critic is retrained on our constraint costs so that it competes on safety rather than only on task failure. Beating it means matching or exceeding its autonomous success on unshifted peg insertion while cutting constraint violations on shifted dynamics. We treat at least a halving of violations at equal success as a meaningful improvement.

The strongest safety-side competitor is a per-step reachability shield in the style of Sim-to-Lab-to-Real. It uses the same critic architecture and training data, but checks every step, aborts the chunk on a predicted violation, and replans. Fairness requires the same ensemble, threshold $\delta$, and number of critic queries per second of robot time.

Two ablations isolate the key ingredient. The first uses a fixed short chunk with no critic. The second uses our critic with a single dynamics model instead of an ensemble. That separates margin-based shortening from uncertainty-based shortening. All methods receive the same hyperparameter search budget, tuned on unshifted dynamics only.

Why not just…?

Why not just shield every step and abort the chunk when needed? This is the most serious threat, because aborting is itself a form of adaptive shortening. The difference is when the decision is made. An abort reacts only after a predicted violation appears mid-chunk. It therefore truncates chunks irregularly, and often in contact phases where commitment matters most. Our method shortens before commitment, and also shortens where dynamics are uncertain even if no violation is predicted. The per-step baseline above settles the question directly. If it matches our success and violation numbers on shifted robosuite tasks, the contribution reduces to the benchmark extension, and we will report it that way.

Why not just randomize dynamics more widely and keep fixed chunks? Broad randomization makes policies conservative in fine contact tasks and offers no protection when the real system falls outside the range. Sim-to-Lab-to-Real stresses that randomized policies still crash on out-of-distribution transitions. Our shifted-dynamics split is designed to fall outside the training range. A wide-randomization fixed-chunk policy trained at equal compute is included, so this hypothesis is tested rather than assumed away.

Why not just use a fixed short chunk? Short chunks restore reactivity but give up the stability benefit. Demystifying Action Space Design reports that chunk-wise delta beats step-wise delta by an average of 10% across tasks, and the chunking theory predicts compounding error grows as chunks shrink. The fixed-short-chunk ablation shows whether adaptivity buys anything over simply picking a small $k$.

Why not just add a large collision penalty and fine-tune? Penalties are known to be fragile. Safe Reinforcement Learning via Constrained Markov Decision Processes shows that small penalties permit catastrophes while large penalties produce frozen, useless robots. In Safety Gymnasium, PPO-Lag cut violations by 98% relative to PPO at a 45% reward cost. Penalty fine-tuning also requires real-world exploration, which is exactly the unsafe step we aim to avoid.

Proposed Experiments

The primary metrics are task success, cumulative constraint cost per episode, and the realized chunk-length distribution. Each is reported separately on in-range and held-out shifted dynamics.

Decisive check 1: does adaptive chunking beat per-step shielding under shift? On robosuite peg insertion and nut assembly, our method should dominate per-step abort-and-replan in success at equal violation rate.

Decisive check 2: does the ensemble actually detect shift? Realized chunk length should fall specifically in states where the shifted dynamics differ most from simulation. We verify this by correlating $k_t$ with the true per-state transition error, which is available in the surrogate setting. If chunk length does not track the mismatch, the ensemble is not detecting shift and the sim-to-real argument fails.

Decisive check 3: unstable dynamics. On Safety-Gymnasium Car Goal and the RoboHive D'Kitty quadruped, chunking is expected to hurt according to Action Chunking and Data Augmentation. Our method should collapse to short chunks without being told to.

Physical study. The final check is a small Franka study through RoboHive with force-limited peg insertion, comparing our method against ThriftyDAgger and fixed chunks. There is no reported Lacuna result for constrained manipulation success on RoboHive, RLBench, or robomimic tool_hang, so these numbers will be new reference points rather than comparisons to published figures.