TL;DR
RPSI-Nav is a recursive self-improvement framework that enables robots to navigate unfamiliar environments more efficiently by refining both their navigation policies and the very logic used to improve them. By closing the loop between navigation outcomes and the "improver" program, the system autonomously diagnoses failures, requests diagnostic observations, and evolves its own repair mechanisms across successive goals.
What Problem Does This Paper Address?
Zero-Shot Object Navigation (ZSON) requires a robot to find objects in a new environment without any prior training on that specific space. In multi-object scenarios, a smart agent should use experience from finding the first object (e.g., a lamp) to find the second (e.g., a pillow) more quickly.
While recent "self-improving" agents can reflect on past trajectories to revise their navigation rules, they typically rely on human-designed, static improvement mechanisms. If the logic used to diagnose a failure is incomplete—for example, if it fixes where the robot looks but ignores how it looks—the agent's ability to improve eventually plateaus. The authors identify a need for "recursive" improvement: the robot shouldn't just improve its navigation; it should improve its ability to improve.
Methods
The authors propose RPSI-Nav (Recursive Policy Self-Improvement for Navigation). The system maintains two distinct executable programs:
- Navigation Program ($P$): Handles location hypotheses, ranking, and viewpoint selection.
- Improvement Program ($R$): Contains rules for diagnosis ($D$), observation acquisition ($Q$), and revision planning ($U$).
The core innovation is the Recursive Loop. When the robot fails or performs inefficiently, the Improvement Program ($R$) analyzes the logs. If the cause of a failure is ambiguous, $R$ can trigger a "diagnostic view"—spending a portion of the movement budget specifically to gather evidence that distinguishes between competing explanations (e.g., "Is my ranking bad?" vs. "Is my viewpoint occluded?").
Crucially, feedback from subsequent goals is used to evaluate the revisions themselves. If a previous fix to the navigation program ($P$) proves ineffective, the system uses this as evidence to revise the Improvement Program ($R$). This creates a hierarchy where the agent evolves its own diagnostic and repair rules over time. To ensure stability, RPSI-Nav uses Conditional Inheritance, where updates are only carried forward if they pass "applicability guards" based on the current context and budget.
Datasets
The framework was evaluated on several challenging benchmarks:
- HM3D (Habitat Matterport 3D): A standard benchmark for zero-shot multi-object navigation.
- HM3D-MFMON: A subset specifically designed to test navigation across multiple floors.
- GOAT-Bench: A benchmark for multimodal navigation involving category names, language descriptions, and reference images.
- Real-Robot Evaluation: Qualitative tests performed on a quadruped robot in a real indoor environment.
Key Insights and Ideas
- Meta-Improvement is Key: The "aha" moment is that the logic for fixing code is itself code. By making the improver editable, the system can overcome the limitations of its initial human-designed heuristics.
- Diagnostic Probing: In embodied AI, information isn't free. RPSI-Nav treats "moving to learn why I failed" as a first-class action, distinct from "moving to find the goal."
- Long-Horizon Feedback: A failure at the third goal might reveal that a "fix" applied after the first goal was actually flawed. RPSI-Nav explicitly links these long-term dependencies to refine its meta-rules.
Key Results
RPSI-Nav demonstrated significant performance gains over state-of-the-art baselines:
- HM3D (Sequential Protocol): Improved Success Rate (SR) from 54.20% (OneMap baseline) to 63.14% and Success weighted by Path Length (SPL) from 27.80% to 35.81%.
- Multi-Floor Navigation: On the Cross-Floor-Required subset of HM3D-MFMON, it achieved 15.97% SR, a massive jump from the 0% SR of the base OneMap system.
- Ablation Studies: The researchers found that allowing the improver to evolve ($R$ revising $R$) provided a +3.3 SR and +3.7 SPL boost over a version that only allowed the navigation policy to evolve.
Limitations and Future Work
- Movement Budget Trade-offs: Diagnostic views are powerful but consume the same movement budget as navigation. Finding the optimal balance between "learning" and "doing" remains an open challenge.
- LLM Dependency: The system relies on frozen Large Language Models (like Qwen2.5-Coder) for reasoning and code generation. The quality of self-improvement is inherently bounded by the underlying model's coding and logic capabilities.
- Real-World Dynamics: While tested on a quadruped, the high-level programmatic revisions might need more robust integration with low-level controllers to handle highly dynamic or cluttered real-world environments.
Summary updated