Abstract
Autonomous navigation in complex outdoor environments requires more than just geometric obstacle detection; it necessitates a semantic understanding of the surroundings to distinguish between traversable roads and restricted areas like sidewalks or vegetation. This paper presents a high-performance metric-semantic mapping system that fuses LiDAR, camera, and inertial data. By leveraging GPU acceleration and a Truncated Signed Distance Function (TSDF) representation, the system achieves real-time performance, processing frames in under 7ms. A recursive Bayesian update ensures temporal consistency of semantic labels, while a non-projective distance calculation improves reconstruction accuracy. Experimental results on public benchmarks and custom campus datasets demonstrate the system's efficiency and reliability for large-scale autonomous navigation.
Introduction
Most autonomous robots navigate using "geometric maps"—3D clouds of points or voxels that identify physical obstacles. However, geometry alone is insufficient for complex outdoor navigation. To a LiDAR sensor, a flat paved road, a flat sidewalk, and a patch of flat grass look nearly identical. To follow traffic rules and navigate safely, a robot needs semantic information: it must distinguish between a road meant for vehicles and a sidewalk meant for pedestrians.
Existing metric-semantic mapping systems often struggle with computational speed, scalability to large outdoor environments, and the temporal consistency of 2D labels when projected into 3D space. This research addresses these challenges through a modular, GPU-accelerated framework.
System Architecture and Methods
The proposed system, COBRA, utilizes a LiDAR-Visual-Inertial (LVI) setup to create dense, labeled 3D maps. The architecture is designed for maximum throughput and accuracy.
As illustrated in the system overview, the pipeline consists of several key components:
- State Estimation: An EKF-based LVI odometry tracks the robot's position with high precision.
- Semantic Segmentation: A CNN processes 2D camera images to label pixels (e.g., "road," "vegetation," "building").
- TSDF-Based Mapping: Instead of simple point clouds, the system uses a Truncated Signed Distance Function (TSDF) to represent surfaces, allowing for sub-voxel resolution and smoother meshes.
- GPU Acceleration: The entire mapping pipeline—from point cloud processing to mesh updates—is implemented in parallel on the GPU, keeping processing time under 7ms per frame.
- Recursive Bayesian Update: To handle noisy 2D labels, a Bayesian filter tracks the probability of labels for every 3D voxel over time, ensuring that transient errors do not corrupt the global map.
Experimental Results
The system was evaluated on established benchmarks (SemanticKITTI, SemanticUSL) and a custom campus dataset comprising 15 sequences with diverse terrains.
Performance and Accuracy
On a desktop (RTX 3080Ti), the system processes frames in 2.4ms to 6.5ms. On embedded platforms like the Nvidia Jetson Orin, it maintains a viable 20-30ms latency. In tests on SemanticKITTI, the system achieved lower Reconstruction Error (RE) and Chamfer Distance (CD) compared to CPU-based baselines like VoxBlox.
Real-World Validation
Deployment on an autonomous vehicle moving at 3m/s confirmed the system's utility. While a robot using only geometric data might attempt to drive across a flat lawn, the semantic layer correctly identifies grass as a non-drivable zone, ensuring safe and rule-compliant navigation.
Discussion and Future Work
A key insight from this work is that semantic labels act as a critical safety layer for traversability analysis. By using a "non-projective" distance calculation that considers local planarity, the system achieves higher reconstruction accuracy than standard projective methods.
However, limitations remain. The system's heavy reliance on GPU memory may require "submap" strategies for city-scale deployment. Additionally, the lack of a global loop closure mechanism means that small odometry drifts can accumulate over very long distances. Future work will focus on integrating loop closure and exploring kernel-based methods to further improve spatio-temporal semantic consistency.
Summary updated