CPNav: Calibrated Vision-Language Navigation for Object-Goal Search in Unseen Environments

Qizhao Chen, Yuanhong Zeng, Shoh Nishino, Anushri DixitarXiv 2026 · arXiv

TL;DR

CPNav is a novel navigation framework that uses Conformal Prediction (CP) to calibrate the often overconfident action scores of Vision-Language Models (VLMs). By statistically filtering out low-confidence actions and maintaining a tree-structured exploration history, CPNav provides formal guarantees for object-goal search, significantly improving success rates and efficiency in unseen environments.

What Problem Does This Paper Address?

Large Vision-Language Models (VLMs) like GPT-4 or Gemini have revolutionized "embodied AI" by providing robots with a form of semantic common sense. They can infer that a "toilet" is likely down a hallway or that a "refrigerator" is in a kitchen. However, these models are primarily trained for text generation or image-text alignment, not for the high-stakes, sequential decision-making required in robotics.

In practice, VLMs are often uncalibrated and overconfident. They might assign a high score to a path that is semantically plausible but physically blocked, or hallucinate the relevance of a distant object. Without a way to quantify and control this uncertainty, VLM-driven robots often get stuck in action loops, perform excessive backtracking, or fail to reach their goals within a reasonable time limit.

Methods

The authors introduce CPNav, a framework that wraps any VLM-based policy in a layer of statistical safety. The pipeline consists of four main stages:

  1. Action Proposer: Using RGB-D data, the robot identifies traversable regions and generates collision-free "motion primitives" (candidate directions to move).
  2. VLM Scoring: Two VLMs are queried. A "Stopping VLM" determines if the goal is reached and provides a global semantic value for the current view. An "Action VLM" scores the candidate motion primitives.
  3. Conformal Calibration: This is the core innovation. Offline, the system performs an "oracle-path search" on a calibration dataset to find the most reliable paths. It uses Conformal Prediction to calculate a threshold ($\hat{q}$) that ensures, with a user-specified probability (e.g., 70%), that the correct oracle action is included in the "admissible" set.
  4. Tree-Structured Exploration: Online, the robot only considers actions that pass the calibrated threshold. If no actions are admissible, it uses a tree-structured history to backtrack to a previous state and try an alternative high-scoring branch.
Fig. 1: Overview of the CPNAV pipeline. (a) The action proposer constructs candidate actions from RGB-D input, prioritizing unexplored regions while enforcing collision safety. (b) The threshold obtained via offline conformal calibration is used to filter out actions whose scores fall below the calibrated bound. (c) The highest-scoring untried action that satisfies the threshold is selected and executed, while the resulting transition is recorded in the exploration tree. (d) Hardware experiments are conducted on a modified Hiwonder MentorPi robot platform.

Datasets

The researchers evaluated CPNav using two major benchmarks in the AI Habitat simulation environment:

  • HM3D v0.2: Used as the primary benchmark for residential environments.
  • MP3D: Used to test "cross-dataset generalizability," ensuring the model works in layouts it hasn't seen during calibration.

For real-world validation, they deployed the system on a modified Hiwonder MentorPi robot equipped with a ZED 2i stereo camera and an NVIDIA Jetson Orin Nano.

Key Insights and Ideas

  • Trajectory-Level Calibration: Instead of calibrating each step in isolation, CPNav uses a "bottleneck" score—the minimum score along a successful path. This ensures that the statistical guarantee holds for the entire trajectory, which is critical for long-horizon tasks.
  • Model-Agnosticism: The CP layer doesn't require access to the VLM's internal weights or training data. It only needs the output scores, making it compatible with proprietary APIs like ChatGPT.
  • Pruning as a Feature: By filtering out low-confidence actions, the robot avoids "semantic traps." The qualitative results show that the uncalibrated baseline often revisits the same areas because the VLM remains stubbornly overconfident about a bad path, whereas CPNav triggers backtracking much earlier.
Comparison of exploration maps: the baseline (left) shows redundant exploration, while CPNav (right) finds the goal efficiently.

Key Results

CPNav consistently outperformed uncalibrated baselines and state-of-the-art methods like STRIVE and WMNav:

  • Simulation (HM3D): Achieved a 6.2% relative improvement in Success Rate (SR) and an 11.4% improvement in Success weighted by Path Length (SPL). It also reduced the mean travel distance by 17.5%.
  • Hardware: The gains were even more pronounced in the real world. CPNav improved SR from 66.7% to 83.3% (a 24.9% relative increase) and reduced travel distance by 30%.
  • Efficiency: CPNav consumed 12.9% fewer VLM tokens than the baseline, as the calibration layer prevented the robot from wasting queries on low-confidence exploration branches.

Limitations and Future Work

While a significant step forward, the authors note several areas for improvement:

  • Orientation-Agnostic States: Currently, tree nodes are defined by spatial proximity. Adding orientation data could improve the accuracy of the exploration history.
  • Sim-to-Real Gap: The calibration threshold is calculated in simulation. While it worked well on hardware, the statistical guarantees technically rely on "exchangeability" between the simulation and the real environment.
  • VLM Stochasticity: Accessing VLMs via APIs introduces noise and latency. Future work will look into making the calibration more robust to the inherent variability of these models.

For those interested in the implementation, the code is available at github.com/practice-lab-ucla/CPnav.

Summary updated

Abstract

Directing a robot to locate a target object in unseen environments remains a central challenge in embodied AI. Vision-Language Model (VLM)-based methods provide rich semantic reasoning but often produce uncalibrated and overconfident action decisions. We introduce CPNav, a navigation framework that combines collision-aware planning with statistically calibrated VLM action scoring for map-free object-goal navigation. From RGB-D input, CPNav constructs a navigability map, generates collision-free motion primitives, and queries two VLMs for goal detection and action scoring. The resulting scores are calibrated offline via conformal prediction (CP) using oracle actions obtained through a tree-structured exploration procedure with backtracking, providing finite-sample guarantees at a user-specified risk level. The CP layer is model-agnostic and can wrap any VLM policy. Online, calibrated VLM scores guide action selection, while a tree-structured exploration history records alternative actions and allows backtracking. We evaluate CPNav in both simulation and real-world hardware settings. In AI Habitat simulation, compared with the uncalibrated baseline method, CPNav achieves relative SR and SPL improvements of 6.2% and 11.4% on HM3D v0.2, and 12.4% and 8.8% on MP3D, respectively. In real-world deployment on a modified Hiwonder MentorPi robot, the corresponding improvements in SR and SPL are 24.9% and 15.2%, respectively. Code is available at https://github.com/practice-lab-ucla/CPnav.
BibTeX
@misc{chen2026cpnav,
  author = {Qizhao Chen and Yuanhong Zeng and Shoh Nishino and Anushri Dixit},
  title = {{CPNav: Calibrated Vision-Language Navigation for Object-Goal Search in Unseen Environments}},
  year = {2026},
  eprint = {2611.10366},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10366},
}
Figure