T2S: Tokenized Skill Scaling for Lifelong Imitation Learning

Hongquan Zhang, Jingyu Gong, Zhizhong Zhang, Xin Tan, Yanyun Qu, Yuan XiearXiv 2025 · DOI · arXiv

Abstract

Lifelong imitation learning in robotics requires balancing the retention of old skills with the capacity to learn new ones. We present Tokenized Skill Scaling (T2S), a framework that tokenizes model parameters and uses natural language instructions to dynamically share and expand these tokens across tasks. T2S achieves state-of-the-art performance in preventing catastrophic forgetting while efficiently scaling to new skills with minimal parameter growth. Evaluated on the LIBERO benchmark, T2S demonstrates near-zero forgetting and high knowledge transfer efficiency.

Introduction

For robots to be useful in the real world, they must continuously acquire new skills over their lifetime. However, sequential training on new tasks typically faces two opposing challenges: Catastrophic Forgetting, where new knowledge overwrites previous weights, and Capacity Saturation, where locking weights to prevent forgetting leaves no room for new learning. Current approaches often treat these as separate issues, using distillation for stability or hierarchical libraries for plasticity. T2S argues that these challenges are internally correlated and can be solved by rethinking how neural networks store and share parameters through tokenization.

Methodology

Tokenized Skill Scaling (T2S) Architecture

The core of T2S is a fundamental alteration of the standard Transformer architecture. Instead of traditional linear layers, T2S introduces the Token-Parameter Attention (Pattention) layer. In this setup, the model's learnable parameters are treated as a pool of discrete tokens. Input tokens from vision and language encoders interact with these parameter tokens via cross-attention. This modularity allows the model to expand its capacity simply by adding new parameter tokens to the global pool.

The T2S architecture integrates multi-modal inputs through a Pattention layer that separates token-token and token-parameter interactions.

Language-Guided Skill Scaling

To prevent parameter explosion as tasks increase, T2S uses natural language instructions (e.g., "pick up the alphabet soup") to query the global token pool. Based on cosine similarity to the language instruction, the model selects a subset of tokens to use for a specific task:

  • Shared Tokens: Reused from previous tasks, these act as "atomic skills" that facilitate knowledge transfer.
  • Task-Specific Tokens: Newly initialized tokens that learn the unique aspects of a novel task.
The similarity matching mechanism routes language embeddings to specific or shared tokens in the key/value pools to balance plasticity and stability.

A hyperparameter ((\mu)) controls the maximum allowance of shared tokens, ensuring a balance between stability (perfectly preserving old knowledge by freezing shared tokens) and plasticity (learning new things via specific tokens).

Experimental Evaluation

The method was evaluated using the LIBERO lifelong robot learning benchmark across three suites of 10 sequential tasks each:

  • LIBERO-OBJECT: Focuses on declarative knowledge (manipulating different objects).
  • LIBERO-GOAL: Focuses on procedural knowledge (following different instructions).
  • LIBERO-SPATIAL: Focuses on spatial adaptability.

Key Results and Insights

T2S was compared against several baselines, including EWC, ER, and M2Distill, showing superior performance in three key metrics:

  1. Near-Zero Forgetting: T2S effectively halted catastrophic forgetting, achieving an average Negative Backward Transfer (NBT) of just 1.0%. In the SPATIAL suite, forgetting was eliminated entirely (0.0%).
  2. High Knowledge Transfer: The model achieved an average Forward Transfer (FWT) of 77.7%, proving that the shared token pool successfully leverages past knowledge to accelerate new learning.
  3. Extreme Parameter Efficiency: By intelligently reusing shared tokens, T2S required only 8.0% of its tokens to be newly trainable on average for lifelong tasks, significantly reducing storage overhead compared to linear-growth models.

Insights on Transfer

The authors observed that procedural knowledge (how to perform an action) transfers more effectively between tasks than declarative knowledge (specific object properties) when sharing tokens. This suggests that tokenization naturally captures the "verbs" of robotic manipulation more readily than the "nouns."

Conclusion and Future Work

T2S demonstrates that tokenizing parameters and using language as a router provides a robust solution to the stability-plasticity dilemma in lifelong learning. While currently validated in simulated environments with frozen perceptual encoders (DINO-v2 and BERT), future work will focus on real-world robotic deployment and exploring end-to-end learning where perceptual representations can also be safely updated over time.

Summary updated

Abstract

The main challenge in lifelong imitation learning lies in the balance between mitigating catastrophic forgetting of previous skills while maintaining sufficient capacity for acquiring new ones. However, current approaches typically address these aspects in isolation, overlooking their internal correlation in lifelong skill acquisition. We address this limitation with a unified framework named Tokenized Skill Scaling (T2S). Specifically, by tokenizing the model parameters, the linear parameter mapping of the traditional transformer is transformed into cross-attention between input and learnable tokens, thereby enhancing model scalability through the easy extension of new tokens. Additionally, we introduce language-guided skill scaling to transfer knowledge across tasks efficiently and avoid linearly growing parameters. Extensive experiments across diverse tasks demonstrate that T2S: 1) effectively prevents catastrophic forgetting (achieving an average NBT of 1.0% across the three LIBERO task suites), 2) excels in new skill scaling with minimal increases in trainable parameters (needing only 8.0% trainable tokens in an average of lifelong tasks), and 3) enables efficient knowledge transfer between tasks (achieving an average FWT of 77.7% across the three LIBERO task suites), offering a promising solution for lifelong imitation learning.
BibTeX
@misc{zhang2025t,
  author = {Hongquan Zhang and Jingyu Gong and Zhizhong Zhang and Xin Tan and Yanyun Qu and Yuan Xie},
  title = {{T2S: Tokenized Skill Scaling for Lifelong Imitation Learning}},
  year = {2025},
  eprint = {2508.01167},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.01167},
}
Figure