UNRealNet: Learning Uncertainty-Aware Navigation Features from High-Fidelity Scans of Real Environments

Samuel Triest, David D. Fan, Sebastian Scherer, Ali-Akbar Agha-Mohammadi

TL;DR

UNRealNet is a "label-free" learning framework that allows small, resource-constrained robots to predict high-fidelity traversability maps from noisy, single-frame LiDAR scans. By training on synthetic data generated from millimeter-accurate laser scans of real-world construction sites, the network learns to "fill in the blanks" caused by occlusions and sensor noise, outperforming traditional mapping baselines by up to 40%.

What Problem Does This Paper Address?

Navigating rugged, unstructured terrain (like a half-finished building or a disaster zone) is a nightmare for small robots. Traditionally, robots build "elevation maps" by aggregating sensor data over time. However, this approach has two major failure points:

  1. Occlusions: If a robot can’t see behind a crate, that area remains a "dead zone" in the map, forcing the planner to take overly cautious or inefficient paths.
  2. Sensing Errors: Cheap LiDARs are noisy, and "odometry drift" (the robot losing track of its exact position) creates ghosting and artifacts in the map.

While simulation is often used to train neural networks to solve these issues, "sim-to-real" gaps remain a hurdle—simulated dirt doesn't always look like real dirt. UNRealNet bridges this gap by using high-fidelity scans of actual environments as the ground truth for training.

Methods

The authors propose a pipeline that turns high-end surveying equipment into a teacher for low-cost mobile robots.

  • Data Generation: They used professional-grade scanners (FARO Focus and Leica BLK360) to create millimeter-accurate 3D models of six real-world environments. They then "virtually" placed a robot in these models to simulate what a noisy, low-resolution LiDAR would see from various vantage points.
  • The Architecture: The core is UNRealNet, a PointPillars-based network. It takes a raw point cloud, organizes it into "pillars" (vertical columns), and uses a UNet backbone to predict a dense map of geometric features like slope, step height, and roughness.
  • Uncertainty-Awareness: Instead of just predicting a single value (e.g., "this slope is 10 degrees"), the network predicts a Gaussian distribution (mean and variance). This allows the robot to know when it is "unsure"—usually in occluded areas or far-away regions.
  • Probabilistic Traversability: They replace traditional hand-tuned cost functions with a probabilistic one. Instead of a human deciding how much "roughness" matters versus "slope," the system calculates the probability that all features are within the robot's physical safety limits.

Datasets

The researchers collected a custom dataset across six unique environments at three active construction sites.

  • Scale: ~30,000 training samples and ~12,000 test samples.
  • Validation: They held out one entire environment (env6) to test generalization. They also validated the model using real-world data from an Ouster OS1 LiDAR mounted on Boston Dynamics Spot and Unitree AlienGo robots.

Key Insights and Ideas

  • The "Inpainting" Superpower: Because the network is trained to predict a full map from a partial scan, it learns environmental priors. For example, it "knows" that floors are usually flat and pillars are vertical. This allows it to accurately predict what’s behind an obstacle (inpainting) before the robot even moves.
  • Robot-Agnosticism: The network predicts geometric features, not a final "can I walk here?" score. This means the same network can be used for a small quadruped and a large wheeled robot; you simply change the "critical values" (e.g., max step height) at deployment time.
  • Real-World Ground Truth > Simulation: By using scans of real construction sites rather than procedurally generated CAD models, the model learns the specific nuances of real-world clutter, wires, and debris that are often missing in simulators.

Key Results

  • Accuracy: UNRealNet outperformed traditional local mapping and standard inpainting algorithms (like Navier-Stokes or Telea) by up to 40% in RMSE (Root Mean Square Error).
  • Occlusion Handling: The network was significantly better at predicting features in "unobserved" cells (areas the LiDAR couldn't see) compared to baselines.
  • Hardware Performance: The system runs at approximately 7Hz on an NVIDIA 3060 Laptop GPU. While slower than some "dumb" mapping methods (13Hz), it is fast enough for legged robots moving at typical speeds (~2m/s).
  • Probabilistic Edge: The uncertainty-aware cost function resulted in a lower Mean Absolute Error (0.23 vs 0.30) when compared to the ground-truth traversability map, proving that reasoning about "known unknowns" leads to safer navigation.

Limitations and Future Work

The authors identify a few areas for improvement:

  • Inference Bottleneck: Currently, the majority of the runtime is spent on preprocessing point clouds in Python. Moving this to C++ or using TensorRT could significantly boost the frame rate.
  • Static Assumption: The current model assumes the world is static. It doesn't yet account for moving people or vehicles, which would require integrating object detection and tracking.
  • Semantic Gap: While the model understands geometry (slopes and bumps), it doesn't understand semantics (e.g., "this is a tall grass" vs. "this is a solid wall"). Future work aims to incorporate semantic labels to help the robot distinguish between traversable vegetation and hard obstacles.

Summary updated

Abstract

Traversability estimation in rugged, unstructured environments remains a challenging problem in field robotics. Often, the need for precise, accurate traversability estimation is in direct opposition to the limited sensing and compute capability present on affordable, small-scale mobile robots. To address this issue, we present a novel method to learn [u]ncertainty-aware [n]avigation features from high-fidelity scans of [real]-world environments (UNRealNet). This network can be deployed on-robot to predict these high-fidelity features using input from lower-quality sensors. UNRealNet predicts dense, metric-space features directly from single-frame lidar scans, thus reducing the effects of occlusion and odometry error. Our approach is label-free, and is able to produce traversability estimates that are robot-agnostic. Additionally, we can leverage UNRealNet's predictive uncertainty to both produce risk-aware traversability estimates, and refine our feature predictions over time. We find that our method outperforms traditional local mapping and inpainting baselines by up to 40%, and demonstrate its efficacy on multiple legged platforms.
BibTeX
@misc{triest2024unrealnet,
  author = {Samuel Triest and David D. Fan and Sebastian Scherer and Ali-Akbar Agha-Mohammadi},
  title = {{UNRealNet: Learning Uncertainty-Aware Navigation Features from High-Fidelity Scans of Real Environments}},
  year = {2024},
  eprint = {2407.08720},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2407.08720},
}
Figure