Introduction
In robot manipulation, the ability to execute complex, multi-stage tasks often hinges on understanding temporal context. However, many current robotic policies operate under the Markov assumption, meaning they rely primarily on the immediate past (1-2 frames) to determine the next action. This leads to a fundamental challenge known as state ambiguity: situations where the current visual input is identical across different stages of a task, yet requires different actions. For example, a robot wiping a table for the second time sees the same visual scene as it did during the first wipe; without memory, it cannot discern whether to continue wiping or return the tool to its holder.
To address this, we introduce PAM (Policy with Adaptive working Memory), a novel visuomotor architecture designed to resolve state ambiguity by maintaining a long-term history of up to 300 frames (~10 seconds) without the computational overhead or overfitting issues typically associated with long-horizon models.
Methodology: Adaptive Working Memory
PAM draws inspiration from the human brain's ability to "recode" raw sensory data into abstract, task-relevant summaries. Instead of attempting to process every pixel of a massive history window, PAM utilizes a specialized architecture to distill information.
Hierarchical Feature Extraction
The system employs a dual-query mechanism to separate immediate physical requirements from long-term context:
- Motion Primitives: One set of tokens focuses on extracting features necessary for immediate physical actions.
- Context Features: A separate set of tokens extracts the "where am I in the task?" signal.
The Context Router
The heart of PAM is the Context Router. It receives context features from the previous 300 frames and uses range-specific queries to compress them into a compact representation. This acts as a bottleneck, forcing the model to retain only the most critical historical cues and preventing the "feature confusion" that often causes jittery movements in other long-history models.
Two-Stage Training and Auxiliary Objectives
To ensure training stability and efficiency, PAM utilizes a two-stage recipe:
- Stage 1: The model is trained to handle basic physical motions.
- Stage 2: The visual encoder is frozen, and the "Temporal Disambiguation" (memory) component is trained using cached features. This allows for efficient parallel training across massive history windows.
Additionally, an Auxiliary Recall Objective is employed. The model is tasked with reconstructing past image features from its compressed memory, ensuring that the stored context remains a faithful and useful record of past events.
Experimental Evaluation
Real-World Benchmarks
PAM was evaluated on seven custom real-world tasks specifically designed to trigger state ambiguity:
- Wipe the Table Twice: Requires counting cycles based on memory.
- Guessing Game: An object is hidden, and the robot must remember its location after a significant delay.
- Buttons in Sequence: Requires remembering which buttons have already been pressed.
Simulation Benchmarks
The model was also tested on Libero-Long, a standard simulation benchmark consisting of 10 long-horizon manipulation tasks, to evaluate its generalizability across diverse environments.
Key Results
Superior Performance and Robustness
In real-world evaluations, PAM achieved an average success rate of 91%. This significantly outperformed other state-of-the-art history-aware models such as LongDP (61%) and MTIL (45%). In the "Guessing Game," PAM maintained high performance even after a 100-frame delay, whereas baseline models failed as they "forgot" the target location.
Efficiency and Generalization
Despite its 300-frame window, PAM maintains an inference speed of over 20Hz on consumer-grade hardware (RTX 4060), making it viable for real-time deployment. On the Libero-Long benchmark, PAM reached an 84.7% success rate, matching or exceeding specialized models like $\pi_0$ and Diffusion Policy.
Interpretability
Visualization of attention maps reveals that PAM effectively utilizes its memory. In cyclic tasks like table wiping, the model specifically attends to the start of previous cycles to determine its current progress, confirming that the context router successfully identifies task-relevant temporal anchors.
Conclusion and Future Work
PAM demonstrates that robots do not need to store every detail of the past; rather, they need an evolving, recoded summary of history. By resolving state ambiguity through an efficient context-routing mechanism, PAM enables more complex and reliable robotic manipulation. Future research will focus on balancing the auxiliary recall objective with motion smoothness and scaling the context router to even larger foundation models for broader "in-the-wild" applications.
Summary updated