Introduction: The Challenge of Large-Scale 3D Mapping
Autonomous driving and urban planning rely heavily on LiDAR-generated 3D point clouds. Segmenting these massive datasets—assigning semantic labels like "road," "building," or "vegetation" to billions of points—presents a significant computational challenge. Researchers typically face a trade-off between data density and scene coverage. High-density crops preserve fine details but lack global context, while downsampling to save memory loses the geometric precision needed for small object recognition.
EyeNet: Mimicking Human Vision
To address this, the authors propose EyeNet, a novel architecture inspired by the human eye's foveal and peripheral vision. Instead of processing a uniform area, EyeNet utilizes a multi-scale sampling strategy that captures a high-density "Central Region" for fine details and a larger, low-density "Peripheral Region" for environmental context.
Human-Inspired Sampling
The input is divided into three distinct point sets:
- Central Region: A high-density sphere focused on the target area.
- Peripheral Region: A larger, sparse area providing broad context.
- Messenger Points: A subset of points shared between both regions, facilitating information flow.
Parallel Processing and Connection Blocks
EyeNet employs two parallel streams based on the RandLA-Net backbone. The "secret sauce" lies in the Connection Blocks, which use self-attentive pooling and channel enhancement to allow the streams to communicate via messenger points. This ensures that local geometric features are informed by the surrounding global context. Finally, a Feature Merging Block with squeeze-and-excitation modules combines the streams for the final prediction.
Key Insights
- Efficiency: By using a sparse periphery, EyeNet covers the same spatial area as standard models while using 60% fewer points. This allows for significantly larger batch sizes during inference.
- Contextual Advantage: The peripheral vision approach is particularly effective for large-scale structures like bridges and rail lines that require a wide receptive field.
- Information Exchange: Ablation studies confirm that the explicit feature exchange in the Connection Blocks is vital, contributing nearly 1% to the overall Mean Intersection over Union (mIoU).
Key Results
EyeNet was evaluated on major benchmarks including SensatUrban, Toronto3D, and DALES. It consistently outperformed existing state-of-the-art methods.
On the SensatUrban dataset, EyeNet achieved an mIoU of 62.30%, a 5% improvement over RandLA++. On Toronto3D, it reached 81.13% mIoU, showing particular strength in identifying difficult classes like road markings. Visual analysis showed that EyeNet produces much cleaner segmentations for large buildings compared to fragmented results from baseline models.
Conclusion and Future Work
EyeNet demonstrates that human-vision-inspired sampling can significantly improve the efficiency and accuracy of 3D semantic segmentation. While currently implemented with a RandLA-Net backbone, the approach is model-agnostic. Future work will explore integrating this strategy with more advanced backbones like Point Transformers and refining performance on extremely sparse aerial datasets.
Summary updated