Look Before You Act: Boosting Pseudo-LiDAR with Online Semantic Embedding

Liangjun Zhang, Tao Song, Tao Jiang, Di Xie, Shiliang PuIROS 2021 · OpenReview

Abstract

Vision-based 3D object detection is a research focus in the field of autonomous driving system. While recently proposed pseudo-LiDAR is a promising solution, its performance is severely restricted by the image-based depth estimator, leading to a considerable performance gap against the LiDAR-based counterparts. In this paper, substantial advances are developed along an orthogonal direction to the previous efforts in the pseudo-LiDAR pipeline. Concretely, we propose a plug- and-play module, called Online Semantic Embedding (OSE), aligning image semantics with the pseudo-LiDAR detection in an end-to-end manner. On the KITTI object detection benchmark, existing stereo-based baselines integrated with our approach show impressive improvements without bells and whistles. Furthermore, we emphasize that OSE works in retrieving the performance under geometric imperfection conditions.
BibTeX
@inproceedings{zhang2021look,
  author = {Liangjun Zhang and Tao Song and Tao Jiang and Di Xie and Shiliang Pu},
  title = {{Look Before You Act: Boosting Pseudo-LiDAR with Online Semantic Embedding}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=j16lUn2rKi},
}

Loading related content...

Figure