Abstract
We present Composable Behavior Embeddings (CBE), a novel framework that compresses complex visual navigation demonstrations into compact, 32-dimensional continuous vectors. By segmenting long trajectories into these "composable" behaviors, robots can perform long-horizon navigation and build topological maps using an order of magnitude less memory than traditional SLAM or behavior-less methods. We demonstrate that CBE achieves high success rates in unseen environments, maintains robustness to obstacles, and operates effectively even with low-resolution visual input.
Introduction
Navigating a robot through complex environments often requires balancing the precision of metric maps with the efficiency of discrete behaviors. Visual SLAM provides accuracy but is memory-intensive and sensitive to image quality or rapid motion. Conversely, discrete behaviors (e.g., "turn left") are efficient but lack the granularity to follow precise, complex paths. The CBE framework addresses this by creating a representation that is compact, continuous, and robust to real-world drift and obstacles.
Methods
The CBE framework consists of three main neural components trained end-to-end: a Behavior Encoder, a Waypoint Generator, and an Attractor Network.
- Behavior Encoder: A recurrent LSTM network that compresses a sequence of demonstration images into a 32-dimensional embedding ($z_D$).
- Waypoint Generator: During execution, this module uses the embedding ($z_D$) and the robot's current visual observation to generate local waypoints, creating a closed-loop system that constantly adjusts to the environment.
- Attractor Network: This network compares the current view with the start and end images of the demonstration to assist in initial alignment and precise goal detection.
To handle long-horizon navigation, the system employs Behavior Segmentation, breaking trajectories into segments at choice points (like intersections) or fixed distances. These segments are then composed or chained together for large-scale navigation across entire buildings.
Experiments and Datasets
The model was trained using the Gibson environment, a high-fidelity simulator of real-world indoor spaces. Training involved 100,000 trajectories from 18 scenes, with evaluation conducted in 5 larger, unseen environments to test generalization. The system utilizes low-resolution 64x64 RGB images with a 120° field of view, demonstrating resilience where traditional SLAM methods often fail due to low resolution.
Key Insights and Results
A key finding is that the model learns an Implicit Visual Odometry. Even without explicit distance training, visualizations of the embedding space show a structured manifold where behaviors cluster by trajectory length and turn direction, suggesting the model internalizes movement geometry.
CBE achieved a success rate of over 95% for single behaviors up to 6 meters. It is exceptionally memory-efficient, requiring only 19 KB per average trajectory, compared to 341 KB for Visual SLAM and 1548 KB for behavior-less approaches. Additionally, the closed-loop waypoint generation allows the robot to autonomously navigate around unseen obstacles and return to its intended path.
Conclusion and Future Work
CBE offers a memory-efficient and robust solution for long-horizon visual navigation by learning a continuous behavior manifold. Future work will focus on improving choice point detection for topological mapping and exploring the integration of natural language instructions directly into the behavior latent space.
Summary updated