GRL-SNAM: Geometric Reinforcement Learning with Path Differential Hamiltonians for Simultaneous Navigation and Mapping in Unknown Environments

Aditya Sai Ellendula, Yi Wang, Minh Son Nguyen, Chandrajit Bajaj

Abstract

We present GRL-SNAM, a geometric reinforcement learning framework for Simultaneous Navigation and Mapping (SNAM) in unknown environments. A SNAM problem is challenging as it needs to design hierarchical or joint policies of multiple agents that control the movement of a real-life robot towards the goal in mapless environments—where the map is not available a priori and must be acquired through sensors. GRL-SNAM differs from preemptive navigation algorithms by relying exclusively on local sensory observations without constructing a global map. Our approach formulates path navigation and mapping as a dynamic shortest path search using controlled Hamiltonian optimization. Sensory inputs are translated into local energy landscapes encoding reachability and obstacle barriers, while policies for sensing, planning, and reconfiguration evolve by updating Hamiltonians. We evaluate GRL-SNAM on 2D navigation tasks, demonstrating that it preserves clearance, generalizes to unseen layouts, and enables high-quality navigation through minimal exploration via local energy refinement.

Introduction

The challenge of Simultaneous Navigation and Mapping (SNAM) in unknown environments typically requires robots to build computationally expensive global maps to plan paths. While Deep Reinforcement Learning (RL) offers a potential solution for mapless navigation, standard algorithms like PPO or SAC often suffer from sample inefficiency and poor generalization to novel obstacle layouts. These methods frequently struggle to balance the physical constraints of the robot with long-term goal-seeking behavior. GRL-SNAM addresses these limitations by treating navigation as a physics-grounded optimization problem rather than a structureless reward-maximization task.

Methodology: Hamiltonian Optimization

Instead of traditional Bellman value functions, GRL-SNAM utilizes Hamiltonian optimization. In this framework, the robot's journey is modeled as an evolving energy landscape where valleys represent safe, goal-directed paths and peaks represent obstacles or deformation constraints.

Multi-Policy Architecture

The system is composed of three integrated sub-modules:

  1. Sensing Policy: Dynamically updates local constraints based on active sensor queries.
  2. Frame Policy: Plans local motion trajectories.
  3. Shape Policy: Manages the physical deformation of the robot (e.g., for squeezing through tight gaps).

Actions are generated by following the natural gradients of this learned "reduced Hamiltonian." This approach allows for lightweight online adaptation; if a collision risk is detected, the system dynamically adjusts energy weights—such as increasing obstacle repulsion—using local feedback like clearance and speed.

Experimental Evaluation

The framework was tested on two distinct 2D tasks:

  • Narrow-Gap Passing: A hyperelastic ring must navigate a workspace with bottlenecks that require the robot to deform to pass.
  • Dungeon Navigation: A point-mass navigation task using complex maze layouts derived from standard benchmarks.

Key Results

GRL-SNAM significantly outperforms standard RL baselines in both efficiency and reliability. In the Dungeon Navigation task, GRL-SNAM achieved an 87.5% success rate, while baselines like PPO and SAC plateaued below 30% success under identical sensing constraints.

Comparison of GRL-SNAM (Ours) against a baseline in a dungeon navigation task. The baseline enters a collision loop and fails at step 81, while GRL-SNAM successfully navigates the complex environment over 1600 steps.

Furthermore, the system demonstrates high efficiency with minimal mapping. In the deformable ring task, it achieved a Success-weighted Path Length (SPL) of ~0.95 while mapping only 10-11% of the environment. The framework also proved resilient to "severe noise," maintaining an 87.1% success rate despite position jitter and sensor hallucinations.

Conclusion

GRL-SNAM demonstrates that embedding physical safety constraints and goals directly into an energy manifold creates more stable and interpretable navigation policies. By focusing on local energy refinement rather than global mapping, the system achieves high-quality navigation with minimal exploration. Future work will focus on extending this geometric approach to 3D environments and richer sensory modalities like RGB-D inputs.

Summary updated

Abstract

We present GRL-SNAM, a geometric reinforcement learning framework for Simultaneous Navigation and Mapping(SNAM) in unknown environments. A SNAM problem is challenging as it needs to design hierarchical or joint policies of multiple agents that control the movement of a real-life robot towards the goal in mapless environment, i.e. an environment where the map of the environment is not available apriori, and needs to be acquired through sensors. The sensors are invoked from the path learner, i.e. navigator, through active query responses to sensory agents, and along the motion path. GRL-SNAM differs from preemptive navigation algorithms and other reinforcement learning methods by relying exclusively on local sensory observations without constructing a global map. Our approach formulates path navigation and mapping as a dynamic shortest path search and discovery process using controlled Hamiltonian optimization: sensory inputs are translated into local energy landscapes that encode reachability, obstacle barriers, and deformation constraints, while policies for sensing, planning, and reconfiguration evolve stagewise via updating Hamiltonians. A reduced Hamiltonian serves as an adaptive score function, updating kinetic/potential terms, embedding barrier constraints, and continuously refining trajectories as new local information arrives. We evaluate GRL-SNAM on two different 2D navigation tasks. Comparing against local reactive baselines and global policy learning references under identical stagewise sensing constraints, it preserves clearance, generalizes to unseen layouts, and demonstrates that Geometric RL learning via updating Hamiltonians enables high-quality navigation through minimal exploration via local energy refinement rather than extensive global mapping. The code is publicly available on \href{https://github.com/CVC-Lab/GRL-SNAM}{Github}.
BibTeX
@misc{ellendula2026grl,
  author = {Aditya Sai Ellendula and Yi Wang and Minh Son Nguyen and Chandrajit Bajaj},
  title = {{GRL-SNAM: Geometric Reinforcement Learning with Path Differential Hamiltonians for Simultaneous Navigation and Mapping in Unknown Environments}},
  year = {2026},
  eprint = {2601.00116},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.00116},
}
Figure