When a Large Language Model (LLM) reads billions of words of text, it learns more than just vocabulary and grammar. It internalizes a deep, structural understanding of sequences: how information flows over time, how long-range dependencies resolve, and how abstract goals decompose into step-by-step plans. A growing body of research explores a provocative question: Can this highly optimized sequential prior be transferred to tasks that have absolutely nothing to do with natural language?
By framing physical control, time-series forecasting, and strategic decision-making as sequence modeling problems, researchers have discovered that text-pretrained models provide a powerful foundation for non-linguistic domains.
The Hidden Universality of Language Models
To understand why language pretraining transfers to non-linguistic domains, we must look at what these models learn beyond syntax. In What do Large Language Models Learn beyond Language?, Avinash Madasu et al. demonstrate that text-pretrained models possess a significant advantage when solving purely symbolic or quantitative tasks, such as mathematical operations and string manipulation. This suggests that large-scale text prediction imparts a generalized inductive bias for symbolic reasoning.
This bias captures fundamental properties of time and order. As Zhenghan Tai et al. show in Language Pretraining Gives Structured Forecasters a Sequential Prior, language pretraining establishes a sequential inductive bias that naturally captures structural patterns like drift and multi-scale dependencies. When applied to time-series forecasting, the model can interpret numerical sequences using structural patterns originally honed on natural language text.
Bootstrapping Reinforcement Learning Trajectories
If language models are inherently adept at processing abstract sequences, they should theoretically excel at modeling the trajectories of reinforcement learning (RL) agents. An RL trajectory—composed of sequences of states, actions, and rewards—shares a fundamental mathematical structure with a sequence of words.
Instead of training an agent's policy from scratch, initializing sequence-based RL agents with pretrained language model weights provides an immediate advantage. In Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning, Ruizhe Shi et al. demonstrate that the sequential modeling capabilities acquired during text pretraining provide a powerful inductive bias for physical motion control.
Similarly, Machel Reid et al. explore this in Can Wikipedia Help Offline Reinforcement Learning?, finding that fine-tuning a text-pretrained model on offline RL datasets yields a massive 3-6x acceleration in training speed alongside state-of-the-art performance. The model does not need to spend millions of steps learning how to track basic temporal dependencies; its language-derived prior already provides that foundation.
Mechanisms of Cross-Domain Transfer
How do we bridge the gap between continuous physical environments and discrete language architectures? Researchers utilize three primary paradigms:
1. Weight Initialization and Parameter-Efficient Fine-Tuning
The decision-making policy is directly initialized with language weights. In Pre-Trained Language Models for Interactive Decision-Making, Ekin Akyürek et al. show that this initialization acts as a scaffold for reasoning and planning, even when the inputs are completely non-linguistic. To make this computationally viable, Hongling Zheng et al. introduce the Decomposed Prompt Decision Transformer in Decomposed Prompt Decision Transformer for Efficient Unseen Task Generalization, achieving high generalization by fine-tuning a mere 1.14% of the total parameters.
2. Frozen Sequence Compressors and Encoders
Instead of updating the language model's weights, the model can be kept completely frozen to act as a stable memory or trajectory encoder. In History Compression via Language Models in Reinforcement Learning, Fabian Paischer et al. utilize a frozen transformer as a universal sequence compressor to solve partially observable tasks without task-specific language inputs. Taking a similar approach, Thomas Schmied et al. show in Retrieval-Augmented Decision Transformer: External Memory for In-context RL that frozen language models can serve as domain-agnostic encoders that map state-action trajectories into a shared embedding space, facilitating efficient memory retrieval.
3. Textualization and Prompting
Alternatively, the environment itself can be adapted to match the language model's native modality. In Prompts and Pre-Trained Language Models for Offline Reinforcement Learning, Denis Tarasov et al. convert continuous vector observations into descriptive, human-readable text strings. This simple prompt engineering allows the language model to process non-textual robotics tasks natively, unlocking state-of-the-art decision-making capabilities.
Why Language Priors Excel Over Other Modalities
One might wonder if pretraining on any sequential data, such as images or video, would yield the same benefits. In On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learning, Shiro Takagi investigates this explicitly, comparing language-pretrained transformers against vision-pretrained ones.
Shiro Takagi demonstrates that language pretraining provides a unique structural "context-like" prior that allows models to navigate state spaces efficiently even when historical context is removed. In contrast, vision-pretrained models suffer from severe optimization hurdles like gradient confusion. Language, by its very nature, forces a model to learn abstract, causal, and compositional relationships, making it uniquely suited for cross-domain sequence transfer.
Generalization and Commonsense in Complex Environments
The ultimate benefit of this cross-domain transfer is generalization. Vanilla reinforcement learning agents struggle with out-of-domain scenarios because they lack "commonsense" world logic. In Learning to Generalize for Sequential Decision Making, Xusen Yin et al. address this by distilling task-specific policies into large language models, allowing the agent to leverage broad background knowledge when encountering novel vocabulary or goals. This linguistic prior is also highly effective in text-heavy environments, as noted by Philip Osborne et al. in A Survey of Text Games for Reinforcement Learning informed by Natural Language.
This capacity for abstract reasoning extends directly to physical and strategic domains:
- Embodied Tasks: In Large Language Models as Generalizable Policies for Embodied Tasks, Andrew Szot demonstrates that adapting frozen LLMs into vision-language policies allows embodied agents to achieve high success rates on completely unseen tasks, showing remarkable robustness to new behaviors.
- Continuous Control: Weichao Mao et al. show in Decision Transformer as a Foundation Model for Partially Observable Continuous Control that large-scale sequence models pretrained on text can generalize to physical control tasks with extreme efficiency, adapting to novel system dynamics after seeing only a minimal number of demonstration trajectories.
- Game Theory: Even in complex game-theoretic settings like multi-agent auctions, Kexin Huang et al. find in Auctionformer: A Unified Deep Learning Algorithm for Solving Equilibrium Strategies in Auction Games that text-pretrained sequence models accelerate convergence when mapping participant distributions to optimal strategic actions.
Ultimately, cross-domain sequence transfer reveals that the boundaries we draw between language, logic, and physical action are largely artificial. The sequential priors developed by reading human language provide a universal toolkit for navigating the structured patterns of the physical world.
Key Foundational Work
A Survey of Text Games for Reinforcement Learning informed by Natural Language (2021) — contributed 2 concept elements to this direction.
Learning to Generalize for Sequential Decision Making (2020) — contributed 1 concept elements to this direction.