Testing Action-Quantization Theory at Benchmark Scale with a Digit-Factored Uniform Grid

Robot policies that are trained like language models first turn continuous motor commands into discrete tokens. A recent theory of this step, Understanding Behavior Cloning with Action Quantization, makes two sharp predictions. First, rounding error in the actions accumulates at most quadratically with the task horizon. Second, learned codebooks, the kind used in practice, are unstable in a way that plain uniform grids are not. That paper tested these claims only on a one-dimensional linear toy system, $x_{t+1}=0.2x_t+0.3u_t$. Meanwhile, the systems with the best results on real manipulation benchmarks use exactly the learned codebooks the theory warns against. The field therefore has a provable story that has never been checked on a real robot task, alongside a working practice that the story says should fail.

We propose to settle this empirically. The tool is a tokenizer chosen so the theory's variables can be turned independently. The theory's own bound implies that the grid resolution needed for a given performance level grows like the square of the horizon. That makes flat uniform binning "pseudo-polynomial" in the required precision: its vocabulary, and the data needed to fill it, grow with the numerical value $1/\epsilon_q$ rather than with its number of digits. We remove this dependence by writing each action coordinate as base-$b$ digits on the same uniform grid. The cells, and therefore the smoothness property the theory relies on, are identical to a fine flat grid. Each prediction, however, is only a $b$-way choice. This lets us sweep the rounding error over orders of magnitude, hold the cell geometry fixed, and compare head-to-head against learned residual codebooks on PushT, Block Push, Franka Kitchen and LIBERO.

Core Claim

The bound for stochastic experts in Cao and Xie is $O(H\sqrt{\log|\Pi|/n} + H^2\epsilon_q)$. Here $H$ is the horizon, $n$ is the number of demonstrations, $|\Pi|$ is the size of the policy class, and $\epsilon_q$ is the worst-case rounding error of the quantizer. To keep the rounding term below a target $\delta$, one needs $\epsilon_q \le \delta/H^2$. Flat uniform binning then needs $K \approx H^2/\delta$ bins per dimension, and its output head grows linearly in $K$. This is the same magnitude dependence that knapsack-style dynamic programs have in their capacity, which KNARsack studies for neural reasoners on discrete problems.

The scheduling literature removes such dependencies by restructuring the state space rather than shrinking it. Skeletons and Minimum Energy Scheduling is one example, and Strongly polynomial algorithm for a class of minimum-cost flow problems with separable convex objectives is another. We borrow only the spirit of that move, not its machinery. A coordinate on a $K=b^L$ grid is emitted as $L$ digits. The head size per dimension drops from $K$ to $bL$, which is polynomial in $\log(1/\epsilon_q)$.

Our claim has three testable parts. The first is that at benchmark scale, the performance loss due to rounding grows with horizon at a rate consistent with, and not exceeding, $H^2\epsilon_q$. The second is that learned residual vector quantizers show measurably higher policy discontinuity under small state perturbations than uniform grids at matched $\epsilon_q$, and that this discontinuity predicts their failures on long-horizon tasks. The third is that a digit-factored uniform grid reaches fine resolutions with the sample efficiency of a coarse grid while keeping the smoothness of a fine one.

Any of the three could come out false, and a false result would be informative. If learned codebooks are not less smooth in practice, the theory's central practical warning is moot for real policies.

Research Program

All tokenizers share one backbone, the MinGPT-style transformer with focal loss used by Behavior Transformers and VQ-BeT. Only the action head changes, so differences can be attributed to the tokenizer.

The digit-factored grid decodes in two stages. The leading digit of each dimension is decoded autoregressively across dimensions, because multimodality such as "go left or go right" lives at the coarse scale and needs joint modeling across dimensions. The remaining digits of all dimensions are then decoded in parallel blocks, conditioned on the coarse digits. This borrows the block-wise parallel decoding idea of FASTer, but with a fixed grid instead of a learned codebook.

The analysis has three parts. First, we restate the Cao and Xie conditions for this factorization. The quantizer's cells are those of the flat grid, so the incremental stability and relaxed total-variation continuity conditions transfer directly. What changes is the policy-class term: the head has $D\,b\,L$ outputs rather than $DK$, where $D$ is the number of action dimensions. Second, we build a measurement of the theory's smoothness condition on neural policies. We call it the local discontinuity score: the expected total-variation distance between a policy's action distributions at an observation and at a slightly perturbed copy of that observation. We compute it on held-out demonstration states, at perturbation scales matched to the drift observed during rollouts. Third, we separate rounding from learning by applying execution-time quantization: a fixed, strong continuous policy has its outputs rounded to a grid at rollout time. This changes $\epsilon_q$ while holding the learned policy constant, which gives the cleanest test of the $H^2\epsilon_q$ term.

Baselines

The naive reference point is flat uniform per-dimension binning with 256 bins, as in RT-1 and Q-Transformer. This is the tokenizer the theory endorses. We run it at bin counts from 16 to 4096 on the same backbone. Beating it means matching its success at its best resolution while staying stable at resolutions where its head is starved of data.

The strongest direct competitor is VQ-BeT's hierarchical residual vector quantizer with continuous offsets. It is the learned codebook the theory flags as non-smooth, and it is the strongest tokenizer that runs on the identical backbone. For it we found no Lacuna-reported success number on PushT or Franka Kitchen. We will report one under our own protocol, along with its published 18 ms per-action hardware latency as the latency bar.

On LIBERO, the strongest existing systems are FASTer, at 97.9% average success, and BEAST, at 80% on LIBERO-Long after 20,000 training steps. Both use larger vision-language backbones. We will not claim to beat them outright. Instead, we will swap our tokenizer into a BEAST-style setup at matched training steps, and count matching 80% on LIBERO-Long at 20,000 steps as parity.

On Franka Kitchen, BeT reports a task entropy of 2.47 against 2.96 for the demonstration data. On Block Push it reports near 100% success. These are the diversity and saturation references for $k$-means with offsets.

Every method gets the same demonstrations, the same backbone size, the same number of gradient steps, and the same hyperparameter search budget of twelve configurations each. Each is evaluated over at least five seeds with fixed rollout counts.

Two ablations isolate our key ingredient. The first is a digit-factored grid whose digits are decoded fully autoregressively; this tests whether parallel fine digits cost accuracy. The second is a residual vector quantizer constrained to axis-aligned uniform residual levels. Its geometry approaches ours while it keeps codebook training, which tells us whether smoothness comes from the grid geometry or from avoiding learning.

Why not just…?

Why not just use flat uniform bins plus a continuous offset, as BeT does? This is the most serious threat, because a regressed offset within each bin drives the effective rounding error toward zero. Our answer is to include it as a baseline and measure its discontinuity score. Offsets can reintroduce averaging inside a bin, and the theory does not cover them. If offsets match our grid on long-horizon success and smoothness, the honest conclusion is that resolution scaling is unnecessary in practice. That would be a useful negative result for the theory's practical reach, and we would report it as such.

Why not just use a diffusion policy and skip discretization? VQ-BeT reports diffusion policies at 573 ms per action on hardware, versus 18 ms for VQ-BeT, and a threefold lower long-horizon success attributed to drift. BEAST reports $\pi_0$ at 20% on LIBERO-Long where BEAST reached 80% at equal steps. More fundamentally, a continuous policy cannot test a theory about quantization. We use one only as the fixed policy in the execution-time quantization probe.

Why not just smooth the learned codebook, for example with a Lipschitz penalty? That is the natural fix the theory suggests. It is plausible, so it is our second ablation rather than an afterthought. If a regularized residual quantizer matches the uniform grid on discontinuity score at the same latency, the contribution narrows to the measurement study, and we would say so.

Proposed Experiments

The first decisive check is horizon scaling. Franka Kitchen naturally splits into one to four completed subtasks, and LIBERO-Spatial and LIBERO-Long give short and long horizons. Under execution-time quantization, we plot the success loss against $H^2\epsilon_q$ across a fourfold range of $\epsilon_q$ and fit the exponent on $H$. An exponent clearly above two would contradict the theory's polynomial-compounding claim on real tasks. An exponent near one would show the bound is loose.

The second check is the smoothness gap. At matched $\epsilon_q$, we compare discontinuity scores for the uniform grid, the digit-factored grid, $k$-means with offsets, and the residual vector quantizer. We then test whether these scores predict per-episode failure on long-horizon tasks better than one-step action error does.

The third check is resolution efficiency. We sweep demonstration counts and resolution on PushT and Block Push, with data subsampled from 10% to 100%. The test is whether flat-grid success collapses at fine resolutions under low data while the digit-factored grid holds. That outcome is the empirical signature of removing the dependence on magnitude.

Throughout, we report wall-clock latency per action on a fixed GPU, using VQ-BeT's 18 ms and BEAST's 19 ms per chunk as reference points. This answers the empirical-gap concern that the asymptotic savings come with hidden constant costs.