M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models

Junxiong Wang, Wen-Ding Li, Daniele Paliotta, Daniel Ritter, Alexander M. Rush, Tri Dao

Abstract

Effective reasoning is crucial to solving complex mathematical problems. Recent large language models (LLMs) have boosted performance by scaling test-time computation through long chain-of-thought reasoning. However, transformer-based models are inherently limited in extending context length due to their quadratic computational complexity and linear memory requirements. In this paper, we introduce a novel hybrid linear RNN reasoning model, M1, built on the Mamba architecture, which allows memory-efficient inference. Our approach leverages a distillation process from existing reasoning models and is further enhanced through RL training. Experimental results on the AIME and MATH benchmarks show that M1 not only outperforms previous linear RNN models but also matches the performance of state-of-the-art Deepseek R1 distilled reasoning models at a similar scale. We also compare our generation speed with a highly performant general purpose inference engine, vLLM, and observe more than a 3x speedup compared to a same size transformer. With throughput speedup, we are able to achieve higher accuracy compared to DeepSeek R1 distilled transformer reasoning models under a fixed generation time budget using self-consistency voting. Overall, we introduce a hybrid Mamba reasoning model and provide a more effective approach to scaling test-time generation using self-consistency or long chain of thought reasoning.

Introduction

Modern reasoning models, such as OpenAI’s o1 and DeepSeek-R1, achieve high performance by scaling test-time compute, allowing the model to generate extensive internal chains of thought (CoT) to navigate complex problems. However, the dominant Transformer architecture faces significant scaling bottlenecks: quadratic computational complexity relative to sequence length and linear growth of the KV cache, which creates massive memory overhead during large-batch inference.

To address these limitations, we present M1, a 3B-parameter hybrid reasoning model based on the Mamba architecture. By utilizing a linear RNN/State Space Model (SSM) foundation, M1 maintains high-level reasoning capabilities while significantly reducing the computational and memory costs associated with long-sequence generation.

Methodology

M1 is a hybrid model that interleaves Mamba layers with a limited number of attention layers (6 layers) to maintain the associative memory strengths of Transformers while gaining the efficiency of RNNs. The training process involves a three-stage pipeline:

  1. Distillation (MambaInLlama): We initialize M1’s weights by distilling a pretrained Llama-3.2-3B model using a reverse KL divergence loss to capture the teacher's primary output modes.
  2. Supervised Fine-Tuning (SFT): The model is trained on standard mathematical datasets (OpenMathInstruct-2) followed by 10 billion tokens of high-quality reasoning traces distilled from DeepSeek-R1.
  3. Reinforcement Learning (RL): We employ Group Relative Policy Optimization (GRPO) to refine reasoning. We found that removing the standard KL penalty and incorporating an entropy bonus improved stability for the Mamba-based architecture.

Key Results and Discussion

M1-3B demonstrates competitive performance against state-of-the-art distilled Transformer models. On the AIME25 benchmark, M1-3B achieved a score of 23.5, slightly outperforming the DeepSeek-R1-Distill-Qwen-1.5B score of 23.0. It also showed significant gains on OlympiadBench, scoring 47.3 compared to 43.3 for the DeepSeek model.

Comparison of M1-3B with state-of-the-art reasoning models across mathematical benchmarks.

The primary advantage of M1 lies in its inference efficiency. When tested on an NVIDIA H100 with a batch size of 512, M1 achieved a 3x throughput increase over a similarly sized Llama-3.2-3B model. This efficiency directly translates to better "test-time compute" scaling. Because M1 can generate more samples in a fixed time window, it achieves higher accuracy through majority voting (Self-Consistency) compared to slower Transformer-based counterparts.

AIME25 Accuracy vs. Inference Time. M1 (red) achieves higher accuracy than the DeepSeek-R1-Distill-Qwen-1.5B (blue) within the same time budget.

Conclusion and Future Work

M1 represents a significant step toward making long-chain reasoning more computationally accessible. By combining the efficiency of Mamba with the reasoning capabilities of distilled models, we achieve state-of-the-art performance with a 3x speedup. Future research will focus on optimizing Mamba kernels within inference engines like vLLM, improving cross-architecture distillation (e.g., from Qwen to Llama-based initializations), and scaling the M1 architecture to larger parameter counts (7B to 70B).

Summary updated

Abstract

Effective reasoning is crucial to solving complex mathematical problems. Recent large language models (LLMs) have boosted performance by scaling test-time computation through long chain-of-thought reasoning. However, transformer-based models are inherently limited in extending context length due to their quadratic computational complexity and linear memory requirements. In this paper, we introduce a novel hybrid linear RNN reasoning model, M1, built on the Mamba architecture, which allows memory-efficient inference. Our approach leverages a distillation process from existing reasoning models and is further enhanced through RL training. Experimental results on the AIME and MATH benchmarks show that M1 not only outperforms previous linear RNN models but also matches the performance of state-of-the-art Deepseek R1 distilled reasoning models at a similar scale. We also compare our generation speed with a highly performant general purpose inference engine, vLLM, and observe more than a 3x speedup compared to a same size transformer. With throughput speedup, we are able to achieve higher accuracy compared to DeepSeek R1 distilled transformer reasoning models under a fixed generation time budget using self-consistency voting. Overall, we introduce a hybrid Mamba reasoning model and provide a more effective approach to scaling test-time generation using self-consistency or long chain of thought reasoning.
BibTeX
@inproceedings{wang2025m,
  author = {Junxiong Wang and Wen-Ding Li and Daniele Paliotta and Daniel Ritter and Alexander M. Rush and Tri Dao},
  title = {{M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models}},
  booktitle = {ER @ NeurIPS},
  year = {2025},
  url = {https://openreview.net/forum?id=bOnhqVefxk},
}
Figure