Abstract
Autonomous navigation in unfamiliar environments typically requires expensive depth sensors and pre-mapped environments. We present RANGER, a monocular zero-shot semantic navigation framework that enables robots to locate specific objects using only a single RGB camera. By integrating 3D foundation models and vision-language models (VLMs), RANGER performs real-time 3D reconstruction and semantic mapping. Crucially, the framework supports "contextual adaptation," allowing it to ingest short walkthrough videos to pre-populate its memory, significantly improving navigation efficiency.
Introduction
Finding a specific object, such as a "coffee machine," in a novel environment is a trivial task for humans but poses a significant challenge for robotic systems. Traditional Object Goal Navigation (ObjectNav) systems often rely on "privileged" information, such as precise depth data from LiDAR or RGB-D sensors and ground-truth pose information from simulators. In real-world scenarios, these sensors are often noisy or prohibitively expensive.
Furthermore, most existing systems lack "In-Context Learning" (ICL) capabilities. While a human can watch a 30-second video of a house and navigate it efficiently, traditional robots typically start from scratch in every new environment. RANGER addresses these limitations by utilizing standard monocular video input for both real-time navigation and prior environmental context, bridging the gap between simulation and the messy real world.
Methods
RANGER employs a multi-stage pipeline to transform a 2D video stream into a searchable 3D semantic map:
- Monocular 3D Reconstruction: RANGER utilizes MASt3R-SLAM to predict 3D pointmaps from RGB frames and estimate the robot's 6-DoF pose in real-time, eliminating the need for dedicated depth hardware.
- Semantic Memory Bank: The system detects objects using GroundingDINO and generates precise masks with MobileSAM. Visual features are extracted using CLIP, enabling "open-vocabulary" understanding of the environment.
- Hierarchical Planning: A high-level VLM-driven planner evaluates the exploration value of different regions, deciding between target-oriented movement and frontier exploration. The Fast Marching Method (FMM) then translates these waypoints into low-level motor commands.
- Contextual Adaptation: RANGER can process an offline "context video" to pre-populate its semantic memory bank. This "mental preview" allows the robot to avoid redundant exploration by creating an initial map before physical movement begins.
Results and Discussion
The framework was evaluated on the HM3D (Habitat-Matterport 3D) benchmark across 279 episodes and deployed on a Unitree G1 humanoid robot in real-world office settings.
Performance Benchmarks
On the HM3D benchmark, RANGER achieved a 42.7% Success Rate (SR) using only RGB input. Notably, this outperformed the L3MVN baseline (39.4%), which utilized privileged depth and pose data. This demonstrates that modern 3D foundation models can effectively replace specialized depth sensors for semantic navigation.
Impact of Contextual Adaptation
The introduction of a short offline context video led to a significant performance leap. The Success Rate increased from 44.0% to 58.0%, while the average path length (steps) required to find an object dropped from 171 to 110. This confirms that visual context can serve as a powerful prior for robotic navigation.
Conclusion
RANGER demonstrates that monocular, zero-shot semantic navigation is feasible and efficient. By leveraging foundation models, it achieves high generalization across diverse environments without scene-specific training. Future work will address computational overhead as memory banks scale, improve robustness in dynamic environments with moving obstacles, and refine scale estimation techniques to overcome the inherent ambiguities of monocular vision.
Summary updated