Abstract
We present GRL-SNAM, a geometric reinforcement learning framework for Simultaneous Navigation and Mapping (SNAM) in unknown environments. A SNAM problem is challenging as it needs to design hierarchical or joint policies of multiple agents that control the movement of a real-life robot towards the goal in mapless environments—where the map is not available a priori and must be acquired through sensors. GRL-SNAM differs from preemptive navigation algorithms by relying exclusively on local sensory observations without constructing a global map. Our approach formulates path navigation and mapping as a dynamic shortest path search using controlled Hamiltonian optimization. Sensory inputs are translated into local energy landscapes encoding reachability and obstacle barriers, while policies for sensing, planning, and reconfiguration evolve by updating Hamiltonians. We evaluate GRL-SNAM on 2D navigation tasks, demonstrating that it preserves clearance, generalizes to unseen layouts, and enables high-quality navigation through minimal exploration via local energy refinement.
Introduction
The challenge of Simultaneous Navigation and Mapping (SNAM) in unknown environments typically requires robots to build computationally expensive global maps to plan paths. While Deep Reinforcement Learning (RL) offers a potential solution for mapless navigation, standard algorithms like PPO or SAC often suffer from sample inefficiency and poor generalization to novel obstacle layouts. These methods frequently struggle to balance the physical constraints of the robot with long-term goal-seeking behavior. GRL-SNAM addresses these limitations by treating navigation as a physics-grounded optimization problem rather than a structureless reward-maximization task.
Methodology: Hamiltonian Optimization
Instead of traditional Bellman value functions, GRL-SNAM utilizes Hamiltonian optimization. In this framework, the robot's journey is modeled as an evolving energy landscape where valleys represent safe, goal-directed paths and peaks represent obstacles or deformation constraints.
Multi-Policy Architecture
The system is composed of three integrated sub-modules:
- Sensing Policy: Dynamically updates local constraints based on active sensor queries.
- Frame Policy: Plans local motion trajectories.
- Shape Policy: Manages the physical deformation of the robot (e.g., for squeezing through tight gaps).
Actions are generated by following the natural gradients of this learned "reduced Hamiltonian." This approach allows for lightweight online adaptation; if a collision risk is detected, the system dynamically adjusts energy weights—such as increasing obstacle repulsion—using local feedback like clearance and speed.
Experimental Evaluation
The framework was tested on two distinct 2D tasks:
- Narrow-Gap Passing: A hyperelastic ring must navigate a workspace with bottlenecks that require the robot to deform to pass.
- Dungeon Navigation: A point-mass navigation task using complex maze layouts derived from standard benchmarks.
Key Results
GRL-SNAM significantly outperforms standard RL baselines in both efficiency and reliability. In the Dungeon Navigation task, GRL-SNAM achieved an 87.5% success rate, while baselines like PPO and SAC plateaued below 30% success under identical sensing constraints.
Furthermore, the system demonstrates high efficiency with minimal mapping. In the deformable ring task, it achieved a Success-weighted Path Length (SPL) of ~0.95 while mapping only 10-11% of the environment. The framework also proved resilient to "severe noise," maintaining an 87.1% success rate despite position jitter and sensor hallucinations.
Conclusion
GRL-SNAM demonstrates that embedding physical safety constraints and goals directly into an energy manifold creates more stable and interpretable navigation policies. By focusing on local energy refinement rather than global mapping, the system achieves high-quality navigation with minimal exploration. Future work will focus on extending this geometric approach to 3D environments and richer sensory modalities like RGB-D inputs.
Summary updated