Abstract
Lifelong imitation learning in robotics requires balancing the retention of old skills with the capacity to learn new ones. We present Tokenized Skill Scaling (T2S), a framework that tokenizes model parameters and uses natural language instructions to dynamically share and expand these tokens across tasks. T2S achieves state-of-the-art performance in preventing catastrophic forgetting while efficiently scaling to new skills with minimal parameter growth. Evaluated on the LIBERO benchmark, T2S demonstrates near-zero forgetting and high knowledge transfer efficiency.
Introduction
For robots to be useful in the real world, they must continuously acquire new skills over their lifetime. However, sequential training on new tasks typically faces two opposing challenges: Catastrophic Forgetting, where new knowledge overwrites previous weights, and Capacity Saturation, where locking weights to prevent forgetting leaves no room for new learning. Current approaches often treat these as separate issues, using distillation for stability or hierarchical libraries for plasticity. T2S argues that these challenges are internally correlated and can be solved by rethinking how neural networks store and share parameters through tokenization.
Methodology
Tokenized Skill Scaling (T2S) Architecture
The core of T2S is a fundamental alteration of the standard Transformer architecture. Instead of traditional linear layers, T2S introduces the Token-Parameter Attention (Pattention) layer. In this setup, the model's learnable parameters are treated as a pool of discrete tokens. Input tokens from vision and language encoders interact with these parameter tokens via cross-attention. This modularity allows the model to expand its capacity simply by adding new parameter tokens to the global pool.
Language-Guided Skill Scaling
To prevent parameter explosion as tasks increase, T2S uses natural language instructions (e.g., "pick up the alphabet soup") to query the global token pool. Based on cosine similarity to the language instruction, the model selects a subset of tokens to use for a specific task:
- Shared Tokens: Reused from previous tasks, these act as "atomic skills" that facilitate knowledge transfer.
- Task-Specific Tokens: Newly initialized tokens that learn the unique aspects of a novel task.
A hyperparameter ((\mu)) controls the maximum allowance of shared tokens, ensuring a balance between stability (perfectly preserving old knowledge by freezing shared tokens) and plasticity (learning new things via specific tokens).
Experimental Evaluation
The method was evaluated using the LIBERO lifelong robot learning benchmark across three suites of 10 sequential tasks each:
- LIBERO-OBJECT: Focuses on declarative knowledge (manipulating different objects).
- LIBERO-GOAL: Focuses on procedural knowledge (following different instructions).
- LIBERO-SPATIAL: Focuses on spatial adaptability.
Key Results and Insights
T2S was compared against several baselines, including EWC, ER, and M2Distill, showing superior performance in three key metrics:
- Near-Zero Forgetting: T2S effectively halted catastrophic forgetting, achieving an average Negative Backward Transfer (NBT) of just 1.0%. In the SPATIAL suite, forgetting was eliminated entirely (0.0%).
- High Knowledge Transfer: The model achieved an average Forward Transfer (FWT) of 77.7%, proving that the shared token pool successfully leverages past knowledge to accelerate new learning.
- Extreme Parameter Efficiency: By intelligently reusing shared tokens, T2S required only 8.0% of its tokens to be newly trainable on average for lifelong tasks, significantly reducing storage overhead compared to linear-growth models.
Insights on Transfer
The authors observed that procedural knowledge (how to perform an action) transfers more effectively between tasks than declarative knowledge (specific object properties) when sharing tokens. This suggests that tokenization naturally captures the "verbs" of robotic manipulation more readily than the "nouns."
Conclusion and Future Work
T2S demonstrates that tokenizing parameters and using language as a router provides a robust solution to the stability-plasticity dilemma in lifelong learning. While currently validated in simulated environments with frozen perceptual encoders (DINO-v2 and BERT), future work will focus on real-world robotic deployment and exploring end-to-end learning where perceptual representations can also be safely updated over time.
Summary updated