Learning-based 3D Occupancy Prediction for Autonomous Navigation in Occluded Environments

Lizi Wang, Hongkai Ye, Qianhao Wang, Yuman Gao, Chao Xu, Fei Gao

Abstract

In autonomous navigation of mobile robots, sensors suffer from massive occlusion in cluttered environments, leaving significant amount of space unknown during planning. In practice, treating the unknown space in optimistic or pessimistic ways both set limitations on planning performance, thus aggressiveness and safety cannot be satisfied at the same time. However, humans can infer the exact shape of the obstacles from only partial observation and generate non-conservative trajectories that avoid possible collisions in occluded space. Mimicking human behavior, in this paper, we propose a method based on deep neural network to predict occupancy distribution of unknown space reliably. Specifically, the proposed method utilizes contextual information of environments and learns from prior knowledge to predict obstacle distributions in occluded space. We use unlabeled and no-ground-truth data to train our network and successfully apply it to real-time navigation in unseen environments without any refinement. Results show that our method leverages the performance of a kinodynamic planner by improving security with no reduction of speed in clustered environments.

The Challenge of Occlusion in Navigation

Autonomous navigation systems rely heavily on sensors like LiDAR and depth cameras. However, these sensors are limited by line-of-sight; they can only perceive the surfaces of objects, leaving a "blind spot" or unknown space behind every obstacle.

Standard planners typically handle this unknown space in one of two ways:

  1. Pessimistic: Treating all unknown space as occupied. This ensures safety but results in overly conservative, slow movement as the robot must constantly stop to verify clear paths.
  2. Optimistic: Treating all unknown space as free. This allows for high speed but risks "surprise" collisions when an obstacle is revealed too late for a robot to maneuver.
Figure 1: Conceptual comparison between optimistic planning (red), which risks collision in unknown space, and predicted planning (green), which anticipates hidden geometry.

To bridge this gap, researchers from Zhejiang University developed OPNet, a system designed to mimic human intuition by predicting the likely shape of hidden environments based on visible context.

Methodology: OPNet and Self-Supervised Learning

The core of this approach is the Occupancy Prediction Network (OPNet), a 3D fully-convolutional network. Unlike many modern 3D tasks that use sparse convolutions, OPNet utilizes dense convolutions. The authors found that in occupancy grids, the "empty" space contains vital contextual information that sparse methods often discard.

Network Architecture

OPNet is built on a U-Net architecture enhanced with Atrous Spatial Pyramid Pooling (ASPP). This allows the model to maintain a wide receptive field, enabling it to understand the broader context of a scene—such as recognizing that a vertical surface likely belongs to a continuous wall rather than a floating obstacle.

Figure 4: The OPNet architecture, featuring a U-Net module and Atrous Spatial Pyramid Pooling to process 3D occupancy grids.

Self-Supervised Data Generation

To train the model without the labor-intensive process of manual 3D labeling, the team utilized the Matterport3D dataset. They developed a self-supervised pipeline where a complete 3D scan serves as the ground truth. By simulating a robot's trajectory and using raycasting to generate a "partial" view of that scan, they created 35,000 training pairs. This ensures the model learns to fill the specific types of occlusions a robot actually encounters in the field.

Double-Layer Mapping

The navigation system maintains two concurrent maps: an Original Map containing raw sensor data and a Predicted Map generated by OPNet. A custom fusion rule merges these layers, allowing the planner to prioritize high-confidence sensor data while using predictions to avoid likely collisions in occluded areas.

Experimental Results

OPNet was evaluated using a DJI Robomaster AI UGV and simulated UAVs. Key performance metrics include:

  • Real-Time Efficiency: The network achieves an inference time of 22.1 ms on a GTX1650, making it over twice as fast as state-of-the-art scene completion models like SG-NN. This speed is critical for deployment on embedded hardware like the NVIDIA Xavier.
  • Accuracy: The model reached a precision of 77.6% and a recall of 81.6% in predicting unknown obstacles.
  • Navigation Success: In UAV simulations, the OPNet-enabled planner achieved a 90-92% success rate, a significant improvement over the 63-70% success rate of traditional conservative planners.
  • Path Smoothness: Qualitatively, robots using OPNet generate smoother trajectories, avoiding the jerky "stop-and-peek" behavior common in occluded environments.

Conclusion and Future Work

OPNet demonstrates that deep learning can effectively provide robots with "spatial intuition," allowing for faster and safer navigation in cluttered indoor spaces. While currently optimized for indoor environments using Matterport3D, future work aims to expand the dataset to include outdoor terrains like forests. Additionally, the team plans to further optimize the network for lighter UAV platforms that rely solely on depth cameras, and explore online learning capabilities to allow robots to adapt to new environments in real-time.

Summary updated

Abstract

In autonomous navigation of mobile robots, sensors suffer from massive occlusion in cluttered environments, leaving significant amount of space unknown during planning. In practice, treating the unknown space in optimistic or pessimistic ways both set limitations on planning performance, thus aggressiveness and safety cannot be satisfied at the same time. However, humans can infer the exact shape of the obstacles from only partial observation and generate non-conservative trajectories that avoid possible collisions in occluded space. Mimicking human behavior, in this paper, we propose a method based on deep neural network to predict occupancy distribution of unknown space reliably. Specifically, the proposed method utilizes contextual information of environments and learns from prior knowledge to predict obstacle distributions in occluded space. We use unlabeled and no-ground-truth data to train our network and successfully apply it to real-time navigation in unseen environments without any refinement. Results show that our method leverages the performance of a kinodynamic planner by improving security with no reduction of speed in clustered environments.
BibTeX
@inproceedings{wang2021learning,
  author = {Lizi Wang and Hongkai Ye and Qianhao Wang and Yuman Gao and Chao Xu and Fei Gao},
  title = {{Learning-based 3D Occupancy Prediction for Autonomous Navigation in Occluded Environments}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=cnub5q4ZnO},
}
Figure