Abstract
This article presents a novel and flexible multitask multilayer Bayesian mapping framework with readily extendable attribute layers. The proposed framework goes beyond modern metric-semantic maps to provide even richer environmental information for robots in a single mapping formalism while exploiting intralayer and interlayer correlations. It removes the need for a robot to access and process information from many separate maps when performing a complex task, advancing the way robots interact with their environments. To this end, we design a multitask deep neural network with attention mechanisms as our front-end to provide heterogeneous observations for multiple map layers simultaneously. Our back-end runs a scalable closed-form Bayesian inference with only logarithmic time complexity. We apply the framework to build a dense robotic map including metric-semantic occupancy and traversability layers. Traversability ground truth labels are automatically generated from exteroceptive sensory data in a self-supervised manner. We present extensive experimental results on publicly available datasets and data collected by a 3D bipedal robot platform and show reliable mapping performance in different environments.
Introduction
Modern autonomous robots require a nuanced understanding of their surroundings that extends beyond simple obstacle detection. To navigate complex, unstructured environments, a robot must identify objects (semantics), assess the walkability of surfaces (traversability), and potentially monitor physical properties like friction or temperature. Traditionally, these attributes are estimated by independent modules, which is computationally expensive and ignores the inherent correlations between environmental features—for instance, the fact that a "road" is almost certainly "traversable."
This paper introduces a unified framework for dense, multilayer Bayesian mapping. By leveraging a multitask deep neural network (DNN) and a scalable Bayesian back-end, the system exploits inter-layer relationships to produce more accurate maps with lower uncertainty than isolated processing would allow.
Methods
The framework is divided into a perception-focused front-end and a spatial-reasoning back-end.
Front-End: Multitask Learning with Attention
The perception front-end utilizes a DNN based on the DeepLabv3+ architecture, modified for multitask learning (MTL). While the network uses a shared encoder to extract general features, it employs separate "heads" for different tasks. To mitigate "negative transfer"—where learning one task degrades performance on another—the authors integrated task-specific attention mechanisms, including Squeeze-and-Excitation (SE) blocks and Residual Adapters (RA).
Self-Supervised Labeling
A significant challenge in training traversability models is the lack of manual labels. The authors developed a self-supervised pipeline that generates ground truth by building local 3D elevation maps from LiDAR or depth data. Geometric traversability (calculated from slope and roughness) is projected back into the 2D image plane and refined using semantic labels to filter out noise, such as sky pixels erroneously marked as traversable.
Back-End: Scalable Bayesian Inference
The mapping back-end performs closed-form Bayesian inference. Semantics are modeled using a Categorical distribution with a Dirichlet prior, while traversability is modeled via a Bernoulli distribution. To maintain real-time performance, the system utilizes sparse kernels and k-d trees, achieving logarithmic time complexity ($O(\log N)$). Crucially, the semantic posterior is used to generate "pseudo-measurements" for the traversability layer, allowing the robot's semantic understanding to improve its physical assessment of the terrain.
Experimental Results
The framework was evaluated across three diverse environments: the KITTI autonomous driving benchmark, the TartanAir simulation dataset, and real-world data collected by "Cassie Blue," a 3D bipedal robot at the University of Michigan.
Key findings include:
- Performance Gains: The MTL network with attention modules outperformed single-task baselines. On the KITTI dataset, the "traversable" class saw a 16.08% improvement in Intersection over Union (IoU) compared to standard models.
- Efficiency: The MTL approach is significantly more resource-efficient, requiring approximately 92M parameters compared to the ~137M parameters needed for three separate single-task networks.
- Uncertainty Reduction: Qualitative analysis demonstrated that leveraging inter-layer correlations (e.g., using semantics to inform traversability) significantly reduces the variance and uncertainty in the resulting maps.
- Real-Time Capability: The Bayesian back-end processed data at speeds consistent with real-time sensor collection (approx. 1.83 sec/scan), making it viable for live deployment.
Discussion and Conclusion
The integration of attention mechanisms proved vital; the ablation study showed a 4.64% performance gain attributable specifically to these modules. Furthermore, the ability to use semantics as a prior for physical attributes allows robots to infer traversability even when geometric data is noisy or absent.
While the current system assumes a static world, future work will focus on incorporating temporal predictions to handle dynamic objects like pedestrians. Additionally, the authors aim to refine traversability models to be robot-specific, accounting for the unique physical capabilities of different platforms, such as bipedal versus wheeled robots. This research represents a significant step toward a single, cohesive mapping formalism that empowers robots with a richer, more reliable understanding of their environments.
Summary updated