Fast Navigation Through Occluded Spaces via Language-Conditioned Map Prediction

Rahul Moorthy Mahesh, O. Goktug Poyrazoglu, Yukang Cao, Volkan IslerarXiv 2025 · DOI · arXiv

Introduction: The "Pace Note" Concept

In the high-stakes world of rally racing, drivers rely on "pace notes"—concise verbal instructions from a co-pilot—to navigate blind corners and complex terrain at speeds far exceeding what their immediate vision would allow. A new research paper, "Fast Navigation Through Occluded Spaces via Language-Conditioned Map Prediction," adapts this concept for autonomous robotics. By integrating simple linguistic cues like "Sharp right turn coming up," the proposed PaceForecaster system allows robots to "hallucinate" hidden environments and plan safer, faster trajectories through occluded spaces.

Overview of the PaceForecaster system: (a) limited sensor range, (b) linguistic 'pace notes' from a co-driver, and (c) the resulting optimized trajectory through an occluded turn.

The Challenge of Occlusion

Autonomous vehicles typically face a strict safety-speed trade-off. When a robot encounters an occlusion—such as a sharp hallway corner or a blind T-junction—it must decelerate significantly because its sensors (like LiDAR) cannot see what lies ahead. Traditional SLAM (Simultaneous Localization and Mapping) methods are reactive, only accounting for geometry that has already been observed. PaceForecaster bridges this gap by using high-level human knowledge to inform low-level geometric planning, effectively extending the robot's "vision" beyond its physical sensor range.

The PaceForecaster Architecture

The system employs a multi-stage neural network architecture designed to convert language into spatial predictions:

  1. Instruction Encoding: Natural language instructions are converted into symbolic "pace notes" (direction, severity, and distance) using an LLM-based front-end.
  2. Level-2 Map Prediction: A U-Net architecture takes the current sensor-derived map (Level-1) and the instruction embedding to predict the Level-2 map—the region likely to be revealed once the occlusion is cleared.
  3. Language Conditioning (FiLM): To fuse spatial data with linguistic instructions, the authors use Feature-wise Linear Modulation (FiLM), applying affine transformations to CNN features based on the language input.
  4. Subgoal Prediction: A second head on the U-Net predicts navigation subgoals within the hallucinated Level-2 map.
  5. Log-MPPI Control: These predictions are fed into a Log-MPPI (Model Predictive Path Integral) controller, which optimizes the robot's physical trajectory in real-time.
Visualization of the predicted Level-2 map and navigation subgoals (orange dots) conditioned on linguistic instructions.

Key Insights: Hallucination as a Planning Tool

A critical insight of this work is that the robot does not merely "follow" a command. Instead, it uses the command to generate a mental model of the hidden environment. This allows the physics-based planner to account for the geometry of a turn before it is physically visible. For instance, knowing a turn is "sharp" allows the robot to adopt a "racing line"—entering the turn wider to maintain higher speed and smoother control, rather than making jerky corrections once the corner is revealed.

Results and Performance

The researchers evaluated PaceForecaster using the Visdiff dataset, which contains complex visibility graphs, and validated it in Gazebo simulations. Key findings include:

  • 36% Performance Improvement: In simulated polygonal environments, the system significantly outperformed baselines that relied solely on local sensor data, particularly at high speeds (3m/s).
  • Ablation Success: Map prediction alone accounted for a 32% gain, while the addition of predicted subgoals provided a further 8% boost.
  • Complex Maze Navigation: In Gazebo tests, the system achieved a 20% average gain in success rates, proving particularly effective in scenarios where traditional planners failed entirely due to occlusion complexity.

Conclusion and Future Directions

PaceForecaster demonstrates that language is a powerful tool for bridging the gap between high-level human intuition and low-level robotic control. While the current model focuses on static geometry, future work aims to incorporate dynamic obstacles and terrain awareness (e.g., "Watch out for mud"). By treating language as a predictive sensor, robots can move closer to the fluid, high-speed navigation seen in human-driven racing.

Summary updated

Abstract

In cluttered environments, motion planners often face a trade-off between safety and speed due to uncertainty caused by occlusions and limited sensor range. In this work, we investigate whether co-pilot instructions can help robots plan more decisively while remaining safe. We introduce PaceForecaster, as an approach that incorporates such co-pilot instructions into local planners. PaceForecaster takes the robot's local sensor footprint (Level-1) and the provided co-pilot instructions as input and predicts (i) a forecasted map with all regions visible from Level-1 (Level-2) and (ii) an instruction-conditioned subgoal within Level-2. The subgoal provides the planner with explicit guidance to exploit the forecasted environment in a goal-directed manner. We integrate PaceForecaster with a Log-MPPI controller and demonstrate that using language-conditioned forecasts and goals improves navigation performance by 36% over a local-map-only baseline while in polygonal environments.
BibTeX
@misc{mahesh2025fast,
  author = {Rahul Moorthy Mahesh and O. Goktug Poyrazoglu and Yukang Cao and Volkan Isler},
  title = {{Fast Navigation Through Occluded Spaces via Language-Conditioned Map Prediction}},
  year = {2025},
  eprint = {2512.21398},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.21398},
}
Figure