TL;DR
ReuseMem introduces a framework that allows robots to record a single visual experience and later "interrogate" that memory to satisfy multiple, different manipulation goals. By training a learned Reader to select task-relevant historical evidence based on how well it supports expert-action prediction, the system achieves a 24.5 percentage point improvement in joint goal completion over standard uniform memory baselines.
What Problem Does This Paper Address?
In human-robot collaboration, a robot might observe a scene—like someone putting a USB drive into one of several identical cases—long before it receives a specific request. When a request finally arrives (e.g., "Bring me the USB case" or "Put that case back where you found it"), the robot must retrieve different "facts" from the same past observation.
The core challenge is goal-agnostic recording vs. goal-aware retrieval. Most existing memory systems either store everything (inefficient) or are trained for specific tasks (inflexible). Furthermore, a robot's memory must remain useful even after the environment has changed—for instance, if the cases have been moved since they were first observed. ReuseMem tackles how to efficiently store these experiences and, more importantly, how to learn which specific moments from the past are "evidence" for a current action.
Methods
ReuseMem splits the memory process into three distinct, efficient stages:
- The Writer (Goal-Independent): As the robot observes, it encodes visual features. To save space, it uses appearance sharing: if a new frame looks very similar to a previously stored "prototype," it simply stores a reference to that prototype along with a unique temporal "causal key." This reduces storage by roughly 33% without losing the temporal order of events.
- The Reader (Goal-Conditioned): When a goal (e.g., "Uncover the blue object") is provided, the Reader searches the memory bank. It doesn't just pick one frame; it selects a set of anchors, references, and supports (see Figure 3) to provide a comprehensive historical context.
- Action-Guided Learning: This is the paper's most novel contribution. Instead of training the Reader using simple labels, it is supervised by expert-action prediction. The system compares different candidate memory selections and favors those that result in a lower loss when trying to predict what an expert robot would do in the current scene.
The retrieved memory is fused with the current observation through a lightweight residual interface and used to adapt a frozen, pretrained foundation policy (MME-VLA) using low-rank adaptation (LoRA).
Datasets
The authors evaluated ReuseMem on both simulated and physical platforms:
- Simulation: Four tasks based on the RoboMME benchmark, designed to test event-relative placement, temporal order, hidden object identity, and transient visual cues (e.g., a light flashing on a specific cube).
- Physical: Two real-world tabletop scenarios using an SO101 robot arm. These involved "Case preparation and return" (identifying hidden contents) and "Toy relocation and restoration" (remembering original positions of objects).
Key Insights and Ideas
- Action as the Ultimate Supervisor: The most effective way to decide what to remember is to see what helps the robot act. By linking retrieval directly to action prediction, the Reader learns to ignore irrelevant visual distractors.
- Separating Appearance from Time: By storing unique visual "prototypes" once but maintaining separate temporal entries, ReuseMem achieves the best of both worlds: the efficiency of compression and the precision of a timeline.
- Memory Persistence through Change: The system demonstrates that historical facts (like where an object was) remain valid even after the robot has moved objects around, provided the fusion mechanism can correctly map those past facts onto the "current" scene.
- Efficiency Matters: Within-call caching of memory projections allows the robot to refine its actions (during the denoising steps of a diffusion policy) without re-reading the entire memory every time, saving valuable milliseconds in real-time loops.
Key Results
- Superior Success Rates: In simulation, ReuseMem achieved a 60.5% All-goal success rate, significantly outperforming "Uniform-Memory" (36.0%) and "No-Memory" (5.5%) baselines.
- Real-World Robustness: In physical trials, it completed all three independent goals in a sequence for 70% of the recordings, compared to just 45% for uniform memory.
- Low Latency: The entire pipeline—from receiving a request to dispatching the first command—takes only 175ms, making it viable for interactive robotics.
- Effective Evidence Selection: Qualitative analysis (Figure 3) shows the Reader successfully learns to "jump" to the exact frames where a case's contents were visible or where an object was first placed.
Limitations and Future Work
While ReuseMem is a significant step forward, the authors note several areas for growth:
- Perceptual Correspondence: The system can struggle if the current scene looks drastically different from the recorded memory (e.g., extreme lighting changes or different camera angles).
- Manipulation Precision: The "lightweight" nature of the action interface means it relies heavily on the base policy's motor skills; it cannot easily fix fundamental flaws in the underlying controller.
- Long-Horizon Scaling: The current experiments use relatively short "experiences" (a few hundred frames). Scaling this to hours of persistent memory would require more aggressive hierarchical retrieval or summarization.
Summary updated