Anchored Task Codes over Goal-Blind Skills

Robots that learn many tasks at once usually receive each task as either an arbitrary ID number or a sentence run through a pretrained language model. The ID tells the robot nothing about how tasks relate to each other. The sentence tells it how the tasks sound alike, which is not the same as how they move alike. "Move the slider left" and "move the slider right" are nearly identical sentences that require opposite motions, and a language-conditioned world model on the CALVIN benchmark is confused by exactly this pair (LIMT).

We propose giving each training task a small learned vector, its task code, which starts at a projection of the task's sentence embedding and is free to drift during reinforcement learning, held back only by a weak pull toward that projection. The code is allowed to influence behavior in exactly one place. It sets the mixing weights over a shared library of low-level motor skills that never see the task. Because the skills are task-blind, the code can only say "which skills, how much." Distances between codes therefore measure behavioral similarity in a vocabulary every task shares. The weak pull toward language keeps that space readable from text, so a new instruction can be placed into it with no extra training.

Core Claim

Routing task information exclusively through a gate over goal-blind primitives turns a language-anchored task code into a behavioral task space. Opposing tasks that language merges are pulled apart by reward. Unseen instructions map through the same projection to codes that reuse existing skills. Both effects depend on the asymmetry, meaning the low-level skills are withheld the task information. An identical anchored code fed to a flat policy should yield a space that is less predictive of which tasks transfer to which. This claim is falsifiable, and the decisive experiment is the flat-policy ablation.

Positioning

Three lines of work supply the pieces. The first supplies the mechanism. Multiplicative compositional policies (MCP) blend Gaussian primitives that see only internal state, using a gate that sees the goal. Behavior priors (Tirumala et al., Information asymmetry in KL-regularized RL) show that withholding goal information from a reference policy makes it capture reusable motion. HO2 reports that goal-aware options collapse, either switching every step or collapsing to a single option. APES shows that priors given full context lose transfer (0.00 versus 0.92 on sparse CorridorMaze). All of these condition the gate on coordinates or numeric task parameters, never on language. MCP retrains its gate per task.

The second line motivates the problem. CARE uses a RoBERTa embedding to attend over a mixture of state encoders. LIMT conditions a world model and a policy on Sentence-BERT. PACMAN clusters LLM embeddings and trains one specialist policy per cluster. All three are flat, and the text embedding reaches every layer that acts. CARE is the closest existing work. Its mixture operates over representations fed to a flat SAC actor, and its context vector is also concatenated into that actor. We instead mix executable action distributions, and task information is barred from them.

The third line supplies the idea that a learned parameter vector can serve as a task fingerprint. SPoT and Task Compass show that learned prompts and prefixes cluster by function and predict transfer. We borrow this for control: the task code is a fingerprint over a frozen-in-meaning skill vocabulary.

Research Program

The policy has $k$ Gaussian primitives $\pi_i(a \mid s_{\text{loc}})$. Each sees robot proprioception and object state but not goal coordinates or task identity. A gate $w(s, z_\tau)$ forms the action distribution as their weighted product, as in MCP. Each training task $\tau$ has a learned code $z_\tau$. A frozen sentence encoder gives the instruction embedding $e_\tau$. Training minimizes

$$ \mathcal{L} = \mathcal{L}_{\text{SAC}} + \lambda \sum_\tau \lVert z_\tau - P e_\tau \rVert^2 , $$

where $\mathcal{L}_{\text{SAC}}$ is the soft actor-critic loss and $P$ is a linear projection learned jointly. The weight $\lambda$ sets how strongly codes are pulled toward language. When $\lambda \to \infty$, $z_\tau = P e_\tau$ and the model reduces to an MCP gate conditioned on language. When $\lambda = 0$, codes are free task IDs and offer no route to unseen tasks. At test time an unseen instruction receives the code $P e_{\text{new}}$ and is executed with no gradient steps.

Primitives and gate are trained jointly from scratch by multi-task RL, with no motion-capture or demonstration pretraining, unlike MCP and APES. Which inputs count as "local" is set by hand: goal coordinates are removed from the Meta-World observation. This is the same manual partition those papers use, and we do not claim to automate it.

The analysis has two parts. First, we measure whether code distances predict pairwise transfer, defined as the speed of fine-tuning from task A's code to task B. Second, we inspect how codes for language-confusable pairs move during training.

Baselines

The naive reference is multi-task SAC with one-hot task IDs. Meta-World reports multi-headed SAC at 85% on MT10, falling to 40% on MT50. LIMT reports MT-SAC at 22.50% on CALVIN. This baseline has no route to unseen tasks.

The strongest flat competitor is CARE. It reports 84% success on MT10 (versus 72% for PCGrad and 73% for Soft Modularization), 36% at 100k steps per task, and 30% zero-shot on the held-out drawer-open and window-open after training on 8 tasks. We take the CARE paper's own numbers as reference, but rerun CARE on the same seeds, step budget, and hyperparameter search budget. Beating it means matching it on MT10, within noise. It also means clearly exceeding its 30% zero-shot figure on the same held-out split, with a gain larger than seed variance across at least five seeds.

On MT50, PACMAN reports about 0.60 success with separate specialists. We count our shared-primitive model as competitive if it reaches comparable success with a single network. We will report parameter counts alongside.

On CALVIN, LIMT reports 52.12% multi-task success, and 24.01% when language is replaced by integer IDs. LIMT is a model-based visual method and ours is model-free. We will therefore rerun flat language-conditioned SAC and our model on identical inputs. We cite LIMT's number only as context, not as a like-for-like target.

Three ablations isolate the key ingredients. The first is a language-gated MCP with a frozen embedding ($\lambda \to \infty$). The second is free codes ($\lambda = 0$). The third, the most important, is a flat policy that receives the identical anchored code $z_\tau$ and the identical loss. This last ablation tests whether the goal-blind primitives, rather than the code-learning trick, produce the behavioral geometry.

Why not just…?

Why not just fine-tune the language encoder end-to-end in a flat policy? End-to-end gradients could also separate "left" from "right." The risk is that, with only 10–50 training sentences, fine-tuning distorts the embedding for unseen instructions. Our anchored code leaves the encoder frozen and lets only a few vectors drift. We will include CARE with an unfrozen encoder as an extra arm. If it matches our zero-shot results, the hierarchy is not earning its place, and we will report that.

Why not just concatenate a one-hot ID with the sentence embedding? That combination resolves the left/right confusion on training tasks trivially. However, the ID dimension carries nothing for unseen tasks, so zero-shot behavior is governed entirely by the raw text embedding, and that is where LIMT's confusion lives. Running this as a baseline on the held-out splits settles the question directly.

Why not just attach the anchored code to a flat policy? This is the real threat. Prefix and prompt work suggests learned task vectors become functional fingerprints in flat models too (Task Compass). Our argument rests on MCP and HO2's evidence that task-aware low levels absorb task-specific logic. If that happens, a flat model's code need not encode behavioral similarity, because the network can reinterpret it per task. The flat-code ablation measures this directly through the correlation between code distance and transfer, and through zero-shot success. If the flat model ties, the core claim fails.

Why not just use PACMAN-style committees? Committees avoid interference but share no motor skills. Every specialist relearns reaching and grasping, and new tasks are assigned to the nearest cluster rather than composed. We compare success at a matched step budget and report total parameters.

Proposed Experiments

Experiments run on Meta-World MT10 and MT50, on CARE's 8-train / 2-held-out zero-shot split, and on a 45/5 split of MT50 tasks evaluated zero-shot. PEARL's 30% on ML45 is few-shot rather than zero-shot and is cited only as context. We also run the six CALVIN tasks, reporting per-task success for move_slider_left and move_slider_right separately. Every arm gets five seeds, equal environment steps, and an equal hyperparameter search budget.

There are four decisive checks. The first concerns the slider pair. Their codes should separate during training, measured as cosine distance relative to the initial $P e_\tau$ distance. Their success should exceed both the frozen-language gate and the flat language-conditioned baseline. The second is the transfer test. Across all MT10 task pairs, the rank correlation between code distance and measured fine-tuning transfer should be higher under asymmetry than under the flat-code ablation. The third is zero-shot success on held-out tasks, which should exceed CARE's rerun under matched conditions. The fourth is a sweep over $\lambda$, which should show zero-shot success peaking at an intermediate value. If it peaks at $\lambda \to \infty$, plain language gating suffices. If no value helps zero-shot, the language anchor is inert.

A known risk is that a linear $P$ fit on only 8–45 tasks will generalize poorly. The 45/5 split is included specifically to test whether more training tasks fix this.