Commit Long Enough to Think: Coupling Action-Chunk Length to Test-Time Compute

Robots that "think harder" at deployment have a timing problem. They might sample many candidate motions and keep the best one, or reason before acting. Every extra unit of deliberation is time during which the world keeps moving and the robot must do something. Current systems settle this with two separate fixed knobs. The first is how many future actions the robot commits to before it looks again, called the action chunk length. The second is how much compute it spends choosing the next chunk. We propose to set both knobs jointly, at every decision, from one quantity: how long the robot can safely act without looking. When a learned critic predicts that committing open-loop is nearly free (free-space reaching, steady cruising), the robot commits to a long chunk and uses the slack to run a large search for the next one. When commitment is costly (contact, a swerving neighbor car), it commits briefly and replans with a small, fast search.

The opportunity matters because the literature has documented both halves of the tension but never closed the loop. Sampling more candidates and picking the best with a verifier reduces action error along a power law in the number of samples (RoboMonkey). The same paper's sampling-and-verification loop runs at only 1.5 Hz on an H100. Real-time execution work shows that inference delays of 100–300 ms produce stale, inconsistent actions unless chunks are carefully stitched (RTC, REMAC). A robot that chooses its commitment horizon so that it exactly covers its next deliberation turns latency from a cost into a budget.

Core Claim

The commitment horizon $\tau$, measured in control steps, and the compute budget $k$ for the next decision, measured in candidate samples, should be chosen together. The choice should maximize the value of committing to the current plan's first $\tau$ actions plus the expected benefit of having $k(\tau)$ samples available at the next replanning point. The deadline constraint ties the two: all deliberation for the next chunk must finish while the current prefix executes.

$$ \tau_t^\star = \arg\max_{\tau \in [\tau_{\min}, \tau_{\max}]} \; Q_\tau\!\left(s_t, a_{t:t+\tau}\right) + \lambda\, g\!\left(k(\tau)\right), \qquad k(\tau) = \left\lfloor \frac{\tau\,\Delta t - \delta}{c} \right\rfloor $$

The symbols are defined as follows. $Q_\tau(s, a_{t:t+\tau})$ is a critic that estimates return when the first $\tau$ actions of a candidate chunk run open-loop and the policy then resumes closed-loop control. $\Delta t$ is the control period, $c$ is the wall-clock cost of drawing and scoring one candidate, and $\delta$ is a fixed safety margin. $g(k)$ is the predicted improvement in the next decision's value from searching over $k$ candidates, and $\lambda$ trades the two terms. We fit $g$ offline in the form $g(k) = -(a k^b + c_0)$. That functional form comes from the error-versus-samples scaling law reported by RoboMonkey, here measured in critic value rather than action error.

The first term captures what Decoupled Q-Chunking (DQC) calls open-loop value bias: committing longer without looking costs value. The second term captures what test-time scaling buys. Their sum makes the temporal-resolution trade-off a per-state decision rather than a hyperparameter.

What is new is the coupling. Prior adaptive-horizon methods choose a horizon but spend constant compute. Prior test-time scaling methods vary or fix compute over a fixed window. Neither treats the time bought by a long commitment as the resource that funds deliberation.

Relation to Prior Work

Four lines of work frame the proposal.

The first motivates it. UA2C is the closest existing system. It samples 32 flow-matching chunks for driving and truncates execution to the prefix on which the samples agree, outperforming fixed-length chunking by 13.17%. UA2C keeps two things fixed: a hard cap of five steps and a constant 32 samples at every replanning point. It also uses sample agreement, not value, as the truncation signal. Our method removes the cap, makes compute a function of commitment, and asks whether value-based truncation beats agreement-based truncation.

The second supplies the critic. DQC trains a critic over long chunks and distills it to short policy chunks, reaching 98% on OGBench cube-triple-100M. Chunk-Guided Q-Learning regularizes a one-step critic toward a chunked critic, but degrades in AntMaze when the chunked critic is noisy. Both fix chunk lengths as training hyperparameters and do no test-time search. We reuse their machinery to train a horizon-conditioned critic $Q_\tau$ and query it at deployment. In effect, the chunk prior becomes a hypothesis checked per state rather than a fixed loss term. CGQ's AntMaze failure is a warning we take seriously: if $Q_\tau$ is miscalibrated at long $\tau$, the scheduler will over-commit. One of our checks targets exactly this.

The third supplies the execution layer and constrains the design. RTC freezes the actions that will run during the inference delay and inpaints the rest with guided flow sampling. REMAC instead trains delay robustness into the weights and reaches 81.2% on real Grasp-Hard versus 75.3% for RTC. Both assume a fixed chunk and a fixed delay. Our scheduler makes the delay variable and known in advance, so RTC-style prefix freezing is the natural way to stitch consecutive chunks.

The fourth adapts duration without search. TARC outputs action-hold durations with a static switch cost. It halves control frequency on an RC car and spikes to 50 Hz on disturbances. OptionZero lets tree search use variable-length options, but only in discrete Atari settings with no wall-clock constraint. These show that duration adaptation and deeper search each help on their own; neither ties one to the other.

Research Program

We would build three components.

The first is a horizon-conditioned critic $Q_\tau$. It is trained offline with DQC-style chunked backups for $\tau$ up to a long ceiling, for example 50 steps, deliberately far beyond UA2C's five. The critic receives $\tau$ as an input, and its target is "execute $\tau$ steps open-loop, then follow the distilled reactive policy."

The second is a candidate generator: a flow-matching chunk policy that proposes full-ceiling-length chunks. Short commitments simply use a prefix of the same proposal.

The third is the scheduler. At each replanning point it scores the current best candidate's prefixes under the objective above, commits $\tau^\star$ steps with RTC-style prefix freezing, and spends exactly $k(\tau^\star)$ samples on the next chunk. Those samples are scored by $Q_\tau$ as the verifier.

Compute is accounted in simulated wall-clock time. Each sample costs a fixed fraction $c$ of a control period, and the environment advances during deliberation. Latency is therefore real in every benchmark, including OGBench, where it is normally ignored. We sweep $c$ to emulate hardware ranging from fast to RoboMonkey-like slow.

Analysis focuses on whether $\tau^\star$ tracks task phase. We expect long commitments in transit and short ones at contact or near hazards. We also measure whether the realized $(\tau, k)$ pairs trace a better success-versus-compute frontier than any fixed pair.

Baselines

The naive reference is synchronous fixed-chunk execution with a single sample. The policy predicts a chunk, the robot waits during inference, then executes. This is the "do nothing" point that RTC and REMAC were built to beat. It shows how much latency costs at each compute price $c$.

The strongest fixed competitor is a tuned fixed pair: a chunk length and a sample count chosen together by grid search. Selection uses the same $Q_\tau$ verifier in the style of RoboMonkey's best-of-$k$ with a learned verifier, and execution uses RTC stitching. It is the strongest because any gain we report could otherwise come from the verifier or the stitching alone. It receives the same data, the same critic, the same mean compute per second, and a tuning budget equal to the number of configurations we spend on $\lambda$ and $\delta$. Beating it requires higher success at matched average compute per second, or equal success at lower compute, across at least two compute prices.

The strongest adaptive-horizon competitor is UA2C-style agreement truncation. It uses 32 samples and truncates at the entropy-agreed prefix. We run it with its native cap of five and with our long ceiling, so that horizon length is not confounded with the method. On SUMO mixed traffic, UA2C reports a 13.17% improvement over fixed-length chunking. Beating it means exceeding that margin over the same fixed-chunk reference in our reimplementation, at equal or lower total compute.

On OGBench, the strongest reported offline numbers are DQC's 98% on cube-triple-100M and 93% on cube-quadruple-100M, both feed-forward with no latency modeled. We first reproduce DQC with latency imposed. Then we compare at matched compute, with the bar being to recover most of DQC's latency-free success when latency is expensive.

Two ablations isolate the coupling. In the first, horizon is adaptive but $k$ is fixed. In the second, $k$ follows the deadline rule but $\tau$ is drawn from the same marginal distribution as $\tau^\star$, ignoring the state. If either ablation matches the full method, the coupling is not the source of the gain. A third ablation replaces the value-gap signal with sample-agreement entropy.

Why not just…?

Why not just tune one good chunk length and sample count? RoboPianist found a single good frequency (20 Hz, with both 10 Hz and 100 Hz failing), so a sweep can work when the task is homogeneous. MDP Playground and TARC's 16-to-50 Hz spikes suggest the best resolution moves with the state. This alternative is the main threat, which is why the tuned fixed pair is our primary baseline. We predict it is competitive on phase-homogeneous Kinetix tasks and loses on tasks mixing transit with contact or sudden disturbances. If it wins everywhere, the proposal fails.

Why not just train delay robustness into the weights, as REMAC does? REMAC handles a known, roughly constant delay without added latency. It cannot increase decision quality when the state is hard, and RoboMonkey's 25% out-of-distribution gain comes precisely from extra samples. The two are complementary. We include a REMAC-trained generator as a variant to test whether our scheduler adds value on top of it.

Why not just run anytime sampling in the background with fixed RTC chunks? This is the strongest simple alternative. Search continuously and take the best candidate when the chunk ends. It uses idle time but cannot lengthen commitment when the world is predictable or shorten it when it is not. We implement it at several fixed chunk lengths. If it matches us at its best chunk length on every task, our contribution reduces to tuning, and we would say so.

Why not just remove UA2C's cap? An uncapped agreement signal measures generator uncertainty, not the cost of not looking. Samples can agree confidently on a bad open-loop plan, and agreement says nothing about how much compute the next step deserves. The entropy-signal ablation tests this directly.

Proposed Experiments

Experiments run on three benchmarks. The first is the Kinetix simulator's 12 dynamic tasks under injected delays, the setting of RTC and REMAC. The file reports no per-task success numbers on Lacuna for Kinetix simulation, only REMAC's qualitative superiority, so we report our own reproductions. The second is OGBench cube-triple and cube-quadruple at 100M transitions, with latency imposed. The third is SUMO/FLOW mixed traffic, for direct comparison with UA2C. The main metrics are success rate (or return), mean and peak compute per second, and cumulative jerk, plotted as success-versus-compute frontiers over a sweep of the sample cost $c$.

The decisive check is a matched-compute comparison against the tuned fixed pair and fixed-chunk anytime sampling. The full method must dominate on mixed-phase tasks. Both ablations (adaptive $\tau$ with fixed $k$, and state-blind $\tau$) must fall short of it; otherwise coupling is not the mechanism. Calibration is the second check. We measure the realized return after committing $\tau$ steps against $Q_\tau$'s prediction across $\tau$, and compare it to the realized over-commitment failure rate. If long-$\tau$ predictions are biased as CGQ's AntMaze result warns, we will add a conservative lower-expectile version of the critic and report both. The third check is phase analysis: $\tau^\star$ should shorten measurably before contacts and disturbances, as TARC's frequency spikes did without any search.