Choosing the Coordinate Frame as a Discrete Action in Fast Distilled Robot Policies

Robot policies learn faster and generalize better when they describe motion relative to something, such as "move 3 cm toward the cup", rather than in absolute workspace coordinates. The problem is that the right "something" changes during a task. Reaching should be expressed relative to the target object. Inserting a held peg should be expressed relative to the fixture. Recovering from a slip should be expressed relative to the gripper. Current systems fix one frame for the whole task. We propose to make the choice of reference frame part of the action itself. At the start of each short chunk of motion, the policy makes a discrete choice of which frame to act in, and a continuous prediction of the chunk's displacements within that frame. This is a hybrid action space: a categorical decision paired with continuous parameters.

To make this usable at control rates, we train an expressive but slow generative teacher over this hybrid space and distill it into a small student network with one output head per frame. Because each head handles a single frame, the student does not need to average across distinct strategies, which is a known failure mode when multi-modal teachers are compressed into unimodal students. A small piece of algebraic machinery borrowed from denotational semantics keeps the frame vocabulary consistent and decides which frames are admissible in a given state. Its role is deliberately limited, and we test whether it earns its place.

Core Claim

The claim is that per-chunk frame selection, distilled into a multi-head real-time student, beats the best fixed-frame policy. The advantage should appear on multi-phase manipulation tasks when objects and fixtures are relocated between training and test. It should come without giving up the error bound that makes chunk-wise relative actions work.

The starting evidence is Demystifying Action Space Design for Robotic Manipulation Policies. Its large real-robot and RoboTwin 2.0 study found that delta actions beat absolute ones. It also found that chunk-wise deltas, where every step in a chunk is measured from the chunk's start, bound error at $O(1)$ per chunk. Step-wise deltas, where each step is measured from the previous predicted step, accumulate error at $O(k)$ over a chunk of length $k$. The chunk-wise form was on average 10% better. That study offers only a binary choice of frame, and it lists adaptive frame switching as an open gap. Our design switches frames only at chunk boundaries, so the $O(1)$ argument still applies within each chunk. That argument comes from measuring all of a chunk's displacements against a single fixed anchor. The new risk is error at the handoff between frames, which we address and measure directly.

Research Program

The action space. The action space is $\mathcal{A} = \bigcup_{f \in \mathcal{F}} \{f\} \times \mathbb{R}^{k \times m}$. Here $\mathcal{F}$ is a small set of candidate anchors (gripper, each detected object, each fixture, and the robot base as an absolute fallback), $k$ is the chunk length, and $m$ is the per-step end-effector delta dimension. Each anchor's frame is translated to the entity's pose and rotated to its orientation, following the local-frame construction of Roto-translated Local Coordinate Frames For Interacting Dynamical Systems, known as LoCS. LoCS assigns each node of a physics graph network its own frame to obtain symmetry under rotation and translation. We reuse its frame construction and its speed-based normalization. We avoid its velocity-as-heading proxy, because that proxy breaks down near zero speed, and instead use object pose estimates where they are available. LoCS fixes one frame per entity for prediction; here the policy chooses among frames for control.

Inferring frame labels. Demonstrations do not come with frame labels, so the teacher must infer them. We segment each demonstration into chunks and represent every chunk in every candidate frame. We then alternate two steps. First, we fit a conditional flow-matching generator over deltas given (state, frame); a flow-matching model is a generator trained to transport noise into data along learned velocity fields. Second, we reassign each chunk to the admissible frame under which the generator's loss is lowest. This is hard expectation-maximization, and it picks out the frame in which the motion looks most regular across demonstrations. The resulting teacher is a categorical frame head plus a per-frame flow model.

Admissibility of frames. Not every frame makes sense in every state. One cannot anchor to an object that is not yet detected, and anchoring to "the held object" implies the gripper is closed. We encode these rules using the information systems of Using Information Systems to Solve Recursive Domain Equations. In that paper, a domain is built from atomic tokens, a consistency predicate over finite sets of tokens, and an entailment relation, and is computed as a least fixed point.

In our case, tokens are facts such as "anchor = object 2", "gripper closed", or "object 2 grasped". From the current perception tokens, we compute the closure under entailment. A frame is admissible if adding its anchor token keeps the set consistent. The same entailment check applied to the predicted end state of a chunk tells us which frames the next chunk may use. This gives a handoff rule: the end pose of chunk $t$, expressed in frame $f_t$, is re-expressed in $f_{t+1}$ and becomes the anchor point of the next chunk.

We do not claim new domain theory. The contribution is a compositional way to specify the vocabulary. Larger vocabularies are generated from per-entity rules rather than from per-task masks written by hand. Whether that matters is an empirical question settled below. The theory papers in the file, such as A Domain-Theoretic Model for a Higher-Order Process Calculus, concern semantic adequacy for process calculi. None of them touches control, and we borrow only the token, consistency, and entailment construction.

The real-time student. The student is a shared multilayer-perceptron trunk with one discrete head over $\mathcal{F}$, masked by admissibility, and one continuous Gaussian head per frame. We evaluate the trunk once and then only the selected head. This avoids the $K$-pass cost of Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spaces, known as MP-DQN. MP-DQN runs one forward pass per discrete action, with the other actions' parameters masked out, to stop parameters of unused actions from corrupting value estimates through what its authors call "false gradients". We keep its central insight, that parameters of unselected modes must not receive gradient, by updating only the chosen head.

Distillation follows the two channels of Adversarial Dual On-Policy Distillation from Expressive Teacher, known as FA-OPD. FA-OPD co-trains a flow-matching teacher with a Gaussian student using two signals. One is a reward derived from the teacher's loss, optimized with on-policy reinforcement learning. The other is a regression of the student's actions onto teacher actions at states the student visits. FA-OPD works only over continuous $\mathbb{R}^n$ actions. We extend both channels to the hybrid space. The reward channel becomes the teacher's joint log-likelihood over frame and deltas. The action channel becomes cross-entropy on the frame plus mean-squared error in the chosen frame's head.

Baselines

The naive reference point is a single-frame chunk-wise delta policy in the robot base frame. This is the configuration recommended by Demystifying Action Space Design, with the same chunk length, trained by behavior cloning with the same Gaussian student architecture. A stronger single-frame reference is a separate policy trained for each candidate anchor, with the best anchor chosen per task on validation data. This oracle-per-task fixed frame is the most honest version of "no switching", and beating it is the core requirement.

The strongest existing method is FA-OPD itself, trained on the same demonstrations with a flow-matching teacher and Gaussian student over continuous chunk-wise deltas in the best fixed frame. The file reports near-100% success on Hand-rotate and Fetch-pick for FA-OPD, success above 0.95 under 1.5x injected noise, and 36 ms versus about 550 ms inference for 40k steps relative to pure flow matching.

We also include the slow flow-matching teacher run directly. It serves as an upper reference on quality and a lower reference on speed.

For primitive-based hybrid control we run RAPS, which replaces raw actions with a library of hand-designed parameterized feedback primitives. The file reports that RAPS solved 43 of 50 Meta-World tasks, against 21 for raw-action reinforcement learning. RAPS learns by reinforcement learning rather than from demonstrations, so we compare it on Meta-World under its own protocol and report it separately rather than pretending the settings match.

All imitation baselines share demonstrations, perception inputs, student size, teacher size, number of gradient steps, and a fixed hyperparameter search budget of equal trial count.

Two ablations isolate the key ingredients. The first keeps the frame-inference teacher but distills it into a single-head student that outputs deltas in a frame given as an input token. This tests whether separate heads matter for avoiding mode averaging. The second replaces the information-system admissibility and handoff rule with a hand-written per-task mask, and with no mask at all. This tests whether the algebraic specification adds anything beyond bookkeeping.

Beating a baseline means a higher mean success rate across tasks, with non-overlapping 95% bootstrap intervals over seeds and evaluation layouts. For the student, it also means inference latency within a factor of 1.5 of the single-head Gaussian student.

Why not just…?

Why not just feed all candidate-frame coordinates into one network and let it work out which matters? This is the most serious threat. A network with all relative vectors as inputs can in principle learn implicit frame selection. Observation Space Matters: Benchmark and Optimization Algorithm shows that relative observations already help with spatial generalization. The difference is on the output side. With multi-frame inputs, the network still emits a delta in one fixed output frame. So when an object moves, the mapping from inputs to outputs must be relearned, which is the kind of entanglement LoCS avoids by canonicalizing outputs. We include this "all-frames-in, base-frame-out" model as a baseline on the relocation splits. If it matches our method there, the switching claim fails and we will say so.

Why not just use fixed primitives, as RAPS does? RAPS gets hybrid structure and speed. However, its primitives are engineered controllers run for 100 to 300 steps, and the file notes this produces staccato motion that cannot adapt within a primitive. Our continuous part is a learned chunk, and our discrete part chooses only the frame, not the behavior. The Meta-World comparison and a path-smoothness metric (mean jerk) show whether learned chunks buy fluidity without losing sample efficiency.

Why not just distill a continuous flow-matching teacher with a consistency or one-step method, and let the teacher's multi-modality carry frame changes implicitly? One-step compressed generators such as Consistency Policy preserve multi-modality at low latency. If they also preserved frame-dependent strategies, explicit frame selection would be unnecessary. The single-head ablation and a mode-averaging diagnostic answer this. The diagnostic is the fraction of student chunks whose endpoint lies farther than a threshold from every teacher sample in that state. Our prediction is that mode averaging concentrates at phase transitions, where frame choice changes.

Why not just a hand-written mask instead of information-system closure? For a fixed small task set, a hand mask may be equivalent. The algebraic version is only justified if it holds up when the anchor set grows or tasks recombine familiar entities in new ways. We test exactly that, and we will drop the machinery if it does not help.

Proposed Experiments

Primary benchmark. The primary benchmark is RoboTwin 2.0, using the ten tasks studied in Demystifying Action Space Design. We report the in-distribution split and a relocation split in which target objects and fixtures are moved outside the training placement range. The file gives no absolute per-task success numbers for that study, only the 10% average chunk-wise advantage, so we reproduce the fixed-frame chunk-wise baseline ourselves before comparing.

Secondary benchmarks. FA-OPD's Fetch-pick and Hand-rotate test whether the hybrid extension costs anything on tasks already near ceiling. Meta-World provides the comparison with RAPS.

Decisive checks. The first decisive check is whether the switching student beats the oracle-per-task fixed frame on the relocation split of multi-phase tasks, while staying within the latency budget. If it does not, the core claim is false.

The second check is the handoff error. We measure end-effector discontinuity at frame switches and test whether success degrades with the number of switches per episode. Degradation would signal that the $O(1)$ per-chunk bound is being broken by error accumulating across chunks.

The third check is interpretability. We test whether the frames chosen by the teacher's inference step align with annotated task phases (reach, grasp, transport, insert). If they do not align, the method may be fitting noise in the frame labels.

The fourth check is a vocabulary-growth experiment for the algebraic component. We train on scenes with two to three entities, then test on scenes with five to six entities, including distractors. We compare the information-system mask against a per-task hand mask written for the training scenes and against no mask.

If the first check passes and the fourth fails, the paper becomes a clean result about frame selection as a hybrid action, with the domain-theoretic specification reported as unnecessary.