Learning-based 3D Occupancy Prediction for Autonomous Navigation in Occluded Environments

Lizi Wang, Hongkai Ye, Qianhao Wang, Yuman Gao, Chao Xu, Fei Gao

TL;DR

Researchers from Zhejiang University have developed OPNet, a lightweight 3D deep neural network that predicts hidden obstacles in a robot's "blind spots." By using a novel self-supervised training method to infer what lies behind occlusions, they allow autonomous drones and ground vehicles to navigate cluttered environments more smoothly and safely than traditional "optimistic" or "pessimistic" planning strategies.

What Problem Does This Paper Address?

In autonomous navigation, sensors like LiDAR and depth cameras suffer from a fundamental limitation: they only see the surfaces of objects. This leaves a massive amount of "unknown space" behind every wall, table, or tree.

Standard navigation planners usually handle this in one of two ways:

  1. Pessimistic/Conservative: Treat all unknown space as occupied. This is safe but slow, as the robot constantly stops to "peek" around corners.
  2. Optimistic: Treat all unknown space as free. This is fast but dangerous, often leading to "surprising" collisions when an obstacle suddenly appears.

The authors aim to bridge this gap by mimicking human intuition—predicting the likely shape of the environment (e.g., "that's probably a solid wall, not a doorway") to plan better trajectories before the sensor even sees the hidden area.

Methods

The paper introduces three core components:

  • OPNet (Occupancy Prediction Network): A 3D fully-convolutional network based on a U-Net architecture. It uses Atrous Spatial Pyramid Pooling (ASPP) to gain a wide receptive field, allowing the model to understand the context of a room (like the relationship between a floor and a wall) to predict missing geometry.
  • Self-Supervised Data Generation: To avoid the "tiring work" of manual labeling, the authors developed a way to generate training data from existing 3D datasets (Matterport3D). They take a complete 3D scan, simulate a robot's trajectory through it, and use raycasting to create a "partial" map of what a robot would see. The original, complete scan then serves as the ground truth.
  • Double-Layer Mapping: The navigation system maintains two maps simultaneously: an Original Map (raw sensor data) and a Predicted Map (OPNet's output). A custom fusion rule combines these layers during pathfinding, allowing the robot to favor certain sensor observations while using predictions to avoid likely collisions.

Datasets

The authors utilized the Matterport3D dataset, which contains building-scale RGB-D scans.

  • Training/Validation: They used 80 buildings (65 for training, 15 for validation), generating approximately 35,000 pairs of partial and complete map blocks.
  • Testing: The system was evaluated in simulated 3D corridors and square rooms, as well as real-world experiments using a DJI Robomaster AI UGV equipped with LiDAR.

Key Insights and Ideas

  • Dense vs. Sparse: Interestingly, the authors found that while sparse convolutions are popular for large-scale 3D tasks, dense convolutions performed better for this specific occupancy task. They hypothesize that the "empty" space in occupancy grids carries vital information that sparse methods might ignore.
  • Speed is a Feature: Unlike State-of-the-Art (SOTA) scene completion models designed for high-fidelity graphics, OPNet is optimized for robotics. It prioritizes inference speed over semantic labeling, making it fast enough for real-time use on embedded hardware like the NVIDIA Xavier.
  • Self-Supervision is Key: By simulating navigation trajectories to create "occlusions," the model learns specifically how to fill in the types of gaps a robot actually encounters, rather than just random holes in a mesh.

Key Results

  • Efficiency: OPNet achieved an inference time of 22.1 ms on a GTX1650, making it at least 2x faster than SOTA models like SG-NN and SSCNet.
  • Accuracy: It reached a precision of 77.6% and a recall of 81.6% in predicting unknown obstacles, significantly outperforming existing scene completion baselines.
  • Navigation Performance: In UAV simulations, the "OPNet approach" achieved a 90-92% success rate, compared to just 63-70% for conservative planners.
  • Smoothness: Qualitatively, the robot avoids "premature turns" and generates much smoother paths because it anticipates the thickness of walls and the layout of corridors before it fully rounds a corner.

Limitations and Future Work

  • Dataset Diversity: The current model was trained primarily on indoor building data. The authors note that the lack of diversity (e.g., not including forests or outdoor terrain) can lead to "negative effects" on prediction quality in those specific environments.
  • Sensor Constraints: The current data generation doesn't fully account for the specific Field of View (FoV) of different sensors, which the authors plan to integrate in the future.
  • Hardware Scaling: While fast, the authors aim to further optimize the network to run on even lighter UAV platforms that only carry depth cameras rather than heavy LiDAR units.
  • Online Learning: The authors suggest that "online learning"—allowing the robot to learn from new environments as it explores them—is a promising future direction.

Summary updated

Abstract

In autonomous navigation, sensors suffer from massive occlusion in cluttered environments, leaving a significant amount of space unknown. In practice, treating the unknown space in optimistic or pessimistic ways both set limitations on planning performance. Therefore, aggressiveness and safety cannot be satisfied at the same time. Mimicking human behavior, in this paper, we propose a method based on deep neural network to predict occupancy distribution of unknown space. Specifically, the proposed method utilizes contextual information of environments and prior knowledge to predict obstacle distributions in the occluded space. Our self-supervised learning method use unlabeled and no-ground-truth data and augments the data by simulating navigation trajectories. Our Occupancy Prediction Network is faster than current SOTA scene completion models and is successfully applied to unseen test environments without any refinement. Results show that our predictor leverages the performance of a kinodynamic planner by improving security with no reduction of speed in cluttered environments.
BibTeX
@inproceedings{wang2021learning,
  author = {Lizi Wang and Hongkai Ye and Qianhao Wang and Yuman Gao and Chao Xu and Fei Gao},
  title = {{Learning-based 3D Occupancy Prediction for Autonomous Navigation in Occluded Environments}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=cnub5q4ZnO},
}

Loading related content...

Figure