BINDER: Instantly Adaptive Mobile Manipulation with Open-Vocabulary Commands

Abstract

Open-Vocabulary Mobile Manipulation (OVMM) requires robots to navigate complex environments and interact with objects based on natural language instructions. However, current systems often suffer from "temporal blindness," where computationally expensive 3D mapping leads to discrete, delayed perception. We present BINDER (Bridging INstant and DEliberative Reasoning), a dual-process framework that provides robots with "continuous sight." By decoupling perception into a fast Video-LLM for constant monitoring (Instant Response Module) and a powerful multimodal LLM for strategic 3D planning (Deliberative Response Module), BINDER allows robots to detect objects en-route and adapt to dynamic changes in real-time. Experimental results show that BINDER significantly outperforms state-of-the-art baselines in success rate and efficiency across diverse real-world settings.

The Challenge of Temporal Blindness

In the field of mobile manipulation, robots are tasked with finding and moving objects in unstructured environments like homes or offices. A major bottleneck in existing OVMM frameworks is the reliance on periodic 3D semantic mapping. Because building these maps is computationally intensive—often taking 10 to 30 seconds—robots typically only update their world view at specific waypoints.

This leads to a phenomenon termed "temporal blindness." If a target object (e.g., a banana) appears in the robot's field of view while it is moving toward a distant exploration goal, the robot remains oblivious until it reaches its next scheduled stop. Similarly, if a human moves an object mid-transit, the robot continues to its now-obsolete destination. These delays cause "cascading failures," resulting in missed opportunities, inefficient paths, and jerky, stop-and-go movement patterns.

Expand

Source, p. 1

The BINDER Framework: Dual-Process Reasoning

Inspired by cognitive dual-process theories (System 1 and System 2 thinking), BINDER splits the robot's reasoning into two specialized modules that operate at different speeds and levels of abstraction:

1. Instant Response Module (IRM - "The Eyes"): This module acts as the robot's reactive system. Powered by a high-frequency Video-LLM (specifically Qwen2.5-VL), it continuously monitors 1-second video clips at 8 frames per second. Rather than building a complex map, it looks for task-relevant triggers or signs of imminent failure. It provides immediate feedback signals: CONTINUE for normal operation, ADJUST for minor manipulation corrections, or REPLAN to alert the higher-level brain.
2. Deliberative Response Module (DRM - "The Brain"): This module handles strategic planning and geometric reasoning. Powered by a multimodal LLM (GPT-4o/5), it performs 3D voxel mapping, selects optimal frontiers for exploration, and generates high-level task sequences.

Bidirectional Coordination

The core innovation of BINDER is the bidirectional communication between these modules. The DRM provides the IRM with a guidance prompt tailored to the current task (e.g., "Look for a banana while navigating"). This focuses the IRM's attention, allowing it to trigger a REPLAN the moment a target is spotted. This allows the robot to divert from its path immediately, grabbing the object mid-navigation rather than completing a redundant trek to a distant waypoint.

Expand

Source, p. 3

Experimental Evaluation and Results

BINDER was evaluated using a Hello Robot Stretch SE3 in three increasingly difficult real-world environments: a controlled office, a studio apartment, and a cluttered three-bedroom home. The tasks involved moving multiple objects under dynamic conditions where object positions were perturbed during execution.

Key Performance Metrics

BINDER demonstrated a substantial leap over strong baselines such as DynaMem and OK-Robot:

• Success Rate (SR): In complex multi-object tasks, BINDER achieved a 0.63 SR, vastly outperforming DynaMem's 0.15 SR.
• Efficiency: In the multi-room apartment, BINDER’s Success weighted by Path Length (SPL) was 0.62, nearly doubling the performance of the best baseline (0.36).
• Speed: By avoiding unnecessary "stop-and-think" cycles, BINDER completed tasks approximately 1.5x faster than traditional mapping-heavy systems.

An ablation study confirmed that the IRM is the primary driver of reliability; removing the continuous monitoring module caused the success rate to drop from 0.80 to 0.40 in dynamic settings.

Discussion and Future Work

While BINDER effectively solves the problem of high-level "temporal blindness," analysis of failure cases indicates that fine-grained manipulation remains a challenge. Most failures occurred during the final "placing" phase, often due to gripper misalignments or objects being dropped.

Future research will focus on integrating faster Video-LLMs to further reduce inference lag and developing more robust low-level controllers for the "last centimeter" of manipulation. Additionally, expanding the framework to handle multi-agent environments where dynamic changes are more frequent and complex is a promising next step.