Online Hierarchical Policy Learning using Physics Priors for Robot Navigation in Unknown Environments

Wei Chen, Yuchen Liu, Alexiy Buynitsky, Ahmed H. QureshiarXiv 2025 · DOI · arXiv

Abstract

Robot navigation in large, complex, and unknown indoor environments is a challenging problem. The existing approaches, such as traditional sampling-based methods, struggle with resolution control and scalability, while imitation learning-based methods require a large amount of demonstration data. Active Neural Time Fields (ANTFields) have recently emerged as a promising solution by using local observations to learn cost-to-go functions without relying on demonstrations. Despite their potential, these methods are hampered by challenges such as spectral bias and catastrophic forgetting, which diminish their effectiveness in complex scenarios. To address these issues, our approach decomposes the planning problem into a hierarchical structure. At the high level, a sparse graph captures the environment's global connectivity, while at the low level, a planner based on neural fields navigates local obstacles by solving the Eikonal PDE. This physics-informed strategy overcomes common pitfalls like spectral bias and neural field fitting difficulties, resulting in a smooth and precise representation of the cost landscape. We validate our framework in large-scale environments, demonstrating its enhanced adaptability and precision compared to previous methods, and highlighting its potential for online exploration, mapping, and real-world navigation.

Introduction

Navigating unknown, large-scale indoor environments requires a robot to balance fine-grained obstacle avoidance with global path planning. Traditional methods often fail this balance; sampling-based algorithms like RRT* struggle in narrow passages at scale, while grid-based methods face exponential memory growth. Recent advancements in Neural Time Fields (NTFields) offer a continuous alternative by learning "cost-to-go" functions through the Eikonal equation. However, global neural representations often suffer from spectral bias—a tendency to overlook high-frequency geometric details—and catastrophic forgetting as the robot explores new areas.

To overcome these limitations, we present Modular-NTFields (mNTFields), a hierarchical framework that modularizes the environment into manageable sub-networks governed by a global navigation graph.

Methodology: The mNTFields Framework

The core innovation of mNTFields lies in its "divide and conquer" approach to spatial representation. The system operates on two distinct levels:

  1. High-Level Global Connectivity: The system builds a sparse navigation graph online. By applying morphological operations to raw occupancy maps, it identifies distinct "rooms" and the topological connections (doorways) between them.
  2. Low-Level Local Planning: Instead of a single global network, mNTFields instantiates a dedicated modular sub-network for each identified room. Each module is a physics-informed neural network (PINN) that solves the Eikonal partial differential equation (PDE) locally.
The mNTFields architecture showing the interplay between exploration, graph updates, and modular path planning.

By integrating Temporal Difference Metric Learning (TDM), the robot can approximate surface normals and obstacle boundaries from local sensor data. This ensures that the learned paths are both high-velocity and collision-free. Furthermore, a backtracking mechanism allows the robot to return to known-safe areas if a new region is deemed "unreachable" due to insufficient local training, enhancing exploration robustness.

Experimental Validation

We evaluated the framework across both simulated and real-world scenarios:

  • Simulation: Eight large-scale environments from the Gibson dataset, featuring up to 42 rooms and 1,000+ square meters.
  • Real-World Deployment: The system was deployed on a Unitree B1 quadruped robot equipped with LiDAR, tasked with navigating a complex university building containing narrow hallways and multiple classrooms.
Real-world navigation results showing the quadruped robot (right) following a path generated by the hierarchical cost map (left).

Results and Discussion

The hierarchical approach yielded significant improvements in both efficiency and reliability:

  • Planning Speed: In real-world trials, path planning took only 0.08 seconds, outperforming RRTConnect (~3s) and grid-based FMM (~0.8s).
  • Success Rate: In the complex Maguayo simulation (31 rooms), mNTFields achieved a 93.5% success rate, a substantial increase over the 66% achieved by the non-hierarchical ANTFields baseline.
  • Scalability: By decoupling global planning (via Dijkstra search on the sparse graph) from local maneuvering (via neural fields), the planning time remains nearly independent of the total environment size.

Conclusion

mNTFields demonstrates that modularity and physics-informed learning can effectively mitigate the inherent weaknesses of global neural fields. By solving the Eikonal equation locally within a hierarchical framework, robots can achieve near-instantaneous planning in vast, unknown spaces without human demonstrations. Future work will focus on adapting these fields for dynamic obstacles and extending the solver to higher-dimensional configuration spaces for complex robotic manipulators.

Summary updated

Abstract

Robot navigation in large, complex, and unknown indoor environments is a challenging problem. The existing approaches, such as traditional sampling-based methods, struggle with resolution control and scalability, while imitation learning-based methods require a large amount of demonstration data. Active Neural Time Fields (ANTFields) have recently emerged as a promising solution by using local observations to learn cost-to-go functions without relying on demonstrations. Despite their potential, these methods are hampered by challenges such as spectral bias and catastrophic forgetting, which diminish their effectiveness in complex scenarios. To address these issues, our approach decomposes the planning problem into a hierarchical structure. At the high level, a sparse graph captures the environment's global connectivity, while at the low level, a planner based on neural fields navigates local obstacles by solving the Eikonal PDE. This physics-informed strategy overcomes common pitfalls like spectral bias and neural field fitting difficulties, resulting in a smooth and precise representation of the cost landscape. We validate our framework in large-scale environments, demonstrating its enhanced adaptability and precision compared to previous methods, and highlighting its potential for online exploration, mapping, and real-world navigation.
BibTeX
@misc{chen2025online,
  author = {Wei Chen and Yuchen Liu and Alexiy Buynitsky and Ahmed H. Qureshi},
  title = {{Online Hierarchical Policy Learning using Physics Priors for Robot Navigation in Unknown Environments}},
  year = {2025},
  eprint = {2510.01519},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.01519},
  howpublished = {arXiv},
}
Figure