TL;DR
Compass 1 is a first-of-its-kind embodied navigation foundation model that unifies language reasoning and video generation to achieve persistent spatial awareness. By introducing a "History Representation Map" and a structured VQA supervision protocol, it enables robots to remember and backtrack through nearly two minutes of visual history, achieving a 79.4% success rate in open-world tasks—crushing the 17.8% achieved by the strongest baselines.
What Problem Does This Paper Address?
Most current embodied navigation systems are "short-sighted." They typically only process a handful of recent video frames, leading to a rapid loss of spatial context. This results in several classic failure modes:
- The "Goldfish" Effect: Robots forget targets they saw just a minute ago, making history backtracking (returning to a previous point) nearly impossible.
- Left-Right Deadlocks: In beyond-the-view exploration, models often get stuck in oscillating turning loops because they don't remember where they just looked.
- Fragile Generalization: Models trained in simulation often fail in the messy real world or when moved to a robot with a different camera height (e.g., moving from a tall humanoid to a short wheeled robot).
Compass 1 tackles these by aiming for persistent navigation—the ability to maintain spatial awareness over long horizons using only monocular RGB video.
Methods
The researchers introduced two primary innovations to the standard Vision-Language-Action (VLA) architecture:
1. History Representation Map (HRM)
Instead of flooding the model with every single video frame, Compass 1 uses Action-Guided Keyframe Selection to pick out critical moments. These keyframes and the robot's trajectory are rendered into a 2D visual canvas called the History Representation Map (HRM). This map is fed into the model as a native image input, allowing the backbone to "see" its own past path and landmarks without architectural changes.
2. Structured VQA Supervision
Simply providing a map isn't enough; the model must be taught to use it. The authors introduced a History Utilization protocol using three types of Visual Question Answering (VQA) tasks during training:
- Task-Stage Grounding: "Which part of the instruction am I doing right now?"
- Exploration Consistency: "Have I already looked to the left?"
- Spatiotemporal Target Retrieval: "Which past keyframe showed the target, and where is it relative to me now?"
3. Unified Architecture
Compass 1 uses a unified multimodal backbone (based on Cosmos3-Edge-4B) that features two heads: an autoregressive language head for reasoning/VQA and a diffusion-based video-action head for generating smooth, continuous trajectories.
Datasets
To train a true foundation model, the team built a massive corpus of over 2,500 hours of video:
- 500+ hours of high-quality, self-collected real-world interactions.
- 2,000+ hours of annotated web videos (YouTube).
They developed a semi-automated pipeline to label this data, using large models to synthesize instructions and trajectories, with humans only intervening to verify "stage boundaries" and filter out noisy data.
Key Insights and Ideas
- The Map is an Image: One of the most elegant insights is treating the navigation map as just another image input. This allows the model to leverage its pre-trained visual reasoning capabilities to understand spatial relationships without needing a custom "map-processing" module.
- VQA as "Map Activation": The authors found that without the VQA tasks, the model tended to ignore the HRM. The VQA protocol acts as a "map activation mechanism," forcing the model to attend to historical context to answer questions correctly.
- Height Invariance: Remarkably, training primarily on data from a 1.0m camera height allowed the model to generalize to robots as low as 0.35m (wheeled robots) and as high as 1.5m (drones). This suggests the model has learned a fundamental understanding of 3D space that transcends specific camera placements.
Key Results
Compass 1 demonstrates a "dominant" performance leap over existing SOTA models like AwareVLN and WNM-3D:
- Overall Success Rate: 79.4% vs. the strongest baseline's 17.8%.
- History Backtracking (HBN): In tasks requiring the robot to return to a previously seen object, Compass 1 jumped from the baseline's 0% to 58%.
- Exploration Boost: A 250% improvement in beyond-the-view navigation tasks.
- Zero-Shot Transfer: The model successfully navigated on four different embodiments (wheeled, quadruped, humanoid, and drone) in diverse environments ranging from quiet offices to busy, pedestrian-dense streets.
Limitations and Future Work
While Compass 1 is a major step forward, the authors note some remaining challenges:
- Persistent Backtracking: While 58% is an "unparalleled leap," it is still lower than the success rates for simpler tasks. Occlusions during the return path can still cause the robot to lose its heading.
- Dynamic Jitter: On platforms with high vibration, like drones or humanoids, the "out-of-distribution" visual noise can slightly degrade performance.
- Future Directions: The team suggests that incorporating more diverse web data and refining the "anchor-and-propagate" grounding strategy could further improve the model's robustness in highly cluttered or dynamic scenes.
Summary updated