Abstract
State Space Models (SSMs) offer a promising alternative to Transformers due to their linear scaling with sequence length. However, they traditionally struggle with precise data retrieval and long-range reasoning. This paper introduces Birdie, a training framework that enhances SSMs through bidirectional context processing and a reinforcement-learning-driven curriculum of specialized objectives. Our results demonstrate that Birdie-trained SSMs can rival Transformers in complex retrieval tasks without sacrificing computational efficiency.
Introduction
The current landscape of large language models is dominated by the trade-off between the high accuracy of Transformers and the efficiency of State Space Models (SSMs). While Transformers excel at capturing complex dependencies, their quadratic scaling makes them prohibitively expensive for very long sequences. Conversely, SSMs like Mamba and Hawk scale linearly but suffer from a "memory bottleneck," often failing to retrieve specific information from their compressed hidden states.
Birdie addresses this limitation not by altering the fundamental architecture of SSMs, but by optimizing the training process. By introducing a dynamic curriculum of retrieval-focused tasks, Birdie teaches SSMs to utilize their memory more effectively.
Methodology
The Birdie framework incorporates three core innovations:
1. Bidirectional Context Processing
Standard SSMs are typically causal, processing information only from left to right. Birdie introduces a bidirectional encoder for the initial prompt. By partitioning the model's internal state into forward and reverse components, the model can "look ahead" during the encoding phase, allowing for a more organized representation of the context before generation begins.
2. Specialized Pre-training Objectives
Moving beyond simple next-token prediction, Birdie utilizes a mixture of denoising tasks designed to stress-test and improve the model's retrieval capabilities. These tasks include selective copying, sentence deshuffling, and span infilling.
3. Dynamic RL Curriculum
To manage the complexity of multiple training objectives, Birdie employs a Multi-Armed Bandit (MAB) framework. This reinforcement learning agent acts as a teacher, adaptively adjusting the mixture of tasks based on the model's real-time performance. This ensures that the model focuses on the most informative tasks at each stage of training, avoiding the suboptimality of fixed-ratio mixtures.
Experimental Results
We evaluated Birdie-trained models on a variety of benchmarks, focusing on retrieval-intensive tasks.
Retrieval Stress Tests
In the Multi-Phone Number Retrieval task, models were required to find specific numbers within a 16,000-token context. Standard SSMs trained with next-token prediction showed a rapid decline in accuracy as the number of items to retrieve increased. In contrast, Birdie-trained SSMs maintained high accuracy, significantly narrowing the performance gap with Transformers.
General Benchmarks
Birdie-trained models also showed substantial improvements on standard reading comprehension tasks. For instance, a Birdie-trained Hawk model achieved 55.8% accuracy on SQuAD V2, a dramatic increase from the 16.6% achieved by the baseline version. Importantly, these gains in retrieval did not come at the expense of general knowledge; Birdie models maintained competitive performance on benchmarks like MMLU.
Conclusion
Birdie demonstrates that the perceived architectural weaknesses of SSMs in data retrieval can be largely mitigated through sophisticated training strategies. By combining bidirectional processing with an adaptive, reward-driven curriculum, Birdie enables efficient SSMs to perform at levels previously reserved for more computationally expensive Transformer models. Future work will explore the scalability of this approach to larger model sizes and even more extensive datasets.
Summary updated