Abstract
Standard imitation learning (IL) methods have achieved considerable success in robotics, yet often rely on the Markov assumption, which falters in long-horizon tasks where history is crucial for resolving perceptual ambiguity. This limitation stems not only from a conceptual gap but also from a fundamental computational barrier: prevailing architectures like Transformers are often constrained by quadratic complexity, rendering the processing of long, high-dimensional observation sequences infeasible. To overcome this dual challenge, we introduce Mamba Temporal Imitation Learning (MTIL). Our approach represents a new paradigm for robotic learning, which we frame as a practical synthesis of World Model and Dynamical System concepts. By leveraging the linear-time recurrent dynamics of State Space Models (SSMs), MTIL learns an implicit, action-oriented world model that efficiently encodes the entire trajectory history into a compressed, evolving state. This allows the policy to be conditioned on a comprehensive temporal context, transcending the confines of Markovian approaches. Through extensive experiments on simulated benchmarks (ACT, Robomimic, LIBERO) and on challenging real-world tasks, MTIL demonstrates superior performance against SOTA methods like ACT and Diffusion Policy, particularly in resolving long-term temporal ambiguities. Our findings not only affirm the necessity of full temporal context but also validate MTIL as a powerful and a computationally feasible approach for learning long-horizon, non-Markovian behaviors from high-dimensional observations.
Introduction
Modern imitation learning (IL) frameworks, such as Action Chunking Transformer (ACT) and Diffusion Policy, typically operate under the Markov assumption—the premise that the optimal next action depends solely on the current observation or a minimal window of recent frames. However, many real-world robotic tasks are inherently non-Markovian. Perceptual ambiguity often arises when different stages of a task yield identical visual observations, requiring the agent to maintain a long-term memory of past actions and states to disambiguate its current context.
While Transformers offer a mechanism for temporal modeling, their quadratic computational complexity ($O(L^2)$) relative to sequence length $L$ makes them prohibitively expensive for long-horizon, high-dimensional video sequences. This creates a bottleneck for scaling robotic learning to complex, multi-stage behaviors.
Methods: Mamba Temporal Imitation Learning (MTIL)
MTIL addresses these challenges by replacing the standard attention-based backbone with Mamba-2, a state-of-the-art State Space Model (SSM).
Architecture and Recurrence
Unlike the self-attention mechanism in Transformers, Mamba-2 utilizes linear-time recurrence ($O(L)$). It maintains a compressed hidden state ($h_t$) that evolves over time, effectively serving as a "belief state" that captures the entire trajectory history. This architecture allows the model to process thousands of timesteps without the memory explosion associated with quadratic attention.
Sequential Training and Action Chunking
To ensure the hidden state $h_t$ learns to evolve correctly, MTIL employs a sequential step-based training procedure. Rather than training on shuffled random windows, the model processes trajectories from start to finish. To maintain efficiency, a batch-parallel training scheme is used to process multiple trajectories simultaneously. Furthermore, MTIL incorporates action chunking, where the model predicts a sequence of future actions at each timestep to ensure smooth and coordinated motion.
Experimental Results
The efficacy of MTIL was evaluated across several simulated benchmarks (ACT/ALOHA, Robomimic, LIBERO) and real-world dual-arm manipulation tasks.
The Necessity of Full History
A key finding of this work is that full temporal context is often a requirement for success in complex tasks. In the ACT "Cube Transfer" task, MTIL achieved a 100% success rate, outperforming the original ACT model (90%). As shown in the learning curves, models conditioned on the full history significantly outperform those limited to short-term (e.g., 10-step) history, demonstrating that long-term memory is critical for high-performance imitation learning.
Resolving Ambiguity in the Real World
In real-world "Sequential Insertion" tasks designed to trigger temporal ambiguity, baseline models like ACT often fail (0% success) because they cannot distinguish between visually similar stages of the task. MTIL, by contrast, achieved a 54% success rate, validating its ability to utilize historical context to resolve such ambiguities. Additionally, MTIL scales gracefully compared to Diffusion Policy, which often encounters memory limits when processing long histories.
Conclusion
MTIL demonstrates that State Space Models like Mamba-2 provide a computationally efficient and effective way to incorporate full trajectory history into imitation learning. By moving beyond the Markov assumption, MTIL enables robots to solve long-horizon tasks that were previously inaccessible to standard attention-based or window-based models. Future work may explore combining Mamba’s temporal encoding with more expressive generative action heads to further enhance performance in multi-modal action spaces.
Summary updated