USING EDGECONV TO IMPROVE 3D OBJECT DETECTION FROM RGB-D DATA

Weiwei Lin, Y. Chen, Chunjin Wang, Jonathan Li2019 · DOI

Abstract

In this paper, we proposed a novel 3D deep learning model for object localization and object bounding boxes estimation. To increase the detection efficiency of small objects in the large scale scenes, the local neighbourhood geometric structure information of objects has been taken into the Edgeconv model, which can operate the original point clouds. We evaluated the 3D bounding box with high resolution in the RGB-D dataset and acquired stable effectiveness even under the sparse points and the strong occlusion. The experimental results indicate that our method achieved the higher mean average precision and better IOU of bounding boxes in SUN RGB-D dataset and KITTI benchmark.

Introduction

Detecting objects in 3D space from RGB-D data is a complex task due to the irregular and unordered nature of point clouds. A common framework, such as F-PointNet, extrudes 2D bounding boxes from images into 3D frustums to isolate relevant point cloud data. However, standard PointNet architectures process points independently, lacking awareness of local geometric structures. This "local blindness" often leads to poor instance segmentation, where background points are incorrectly included in the object set, resulting in inaccurate 3D bounding box estimation.

Comparison of false vs. true segmentation and its impact on bounding box accuracy.

Methodology

The proposed architecture enhances the frustum-based pipeline by integrating EdgeConv layers. Unlike traditional point-wise operations, EdgeConv utilizes a K-Nearest Neighbors (KNN) graph (with $K=20$ for KITTI) to capture relationships between a point and its local neighborhood. The authors optimized this approach by reducing layer depth and removing spatial transforms to maintain computational efficiency.

Architecture Overview

The model follows a multi-stage process:

  1. Frustum Proposal: 2D detectors identify objects in RGB images, which are then projected into 3D frustums.
  2. 3D Instance Segmentation: The core innovation involves using EdgeConv layers to learn local geometric features. By identifying the local shape of an object's surface, the network more effectively distinguishes the target object from clutter or nearby surfaces.
  3. 3D Box Estimation: After segmentation, the refined point set is passed to a regression network to estimate the object's 3D center, dimensions, and orientation.
The proposed network architecture featuring EdgeConv layers and KNN graph construction for 3D instance segmentation.

Key Results and Discussion

The model was evaluated on the KITTI and SUN-RGBD benchmarks. By capturing local geometry, the system demonstrated significant improvements in handling sparse data and occlusions.

  • KITTI Benchmark: The method achieved 84.64% AP for Cars (Easy) and 78.17% AP for Cyclists (Easy). The gains in the Cyclist category highlight the model's ability to define the shape of small, thin objects that are often missed by global-only feature extractors.
  • SUN-RGBD Benchmark: The model reached a mean Average Precision (mAP) of 77.6%, a 3.0% improvement over the baseline. Qualitative analysis showed superior performance in separating closely placed objects, such as adjacent tables in an indoor scene.

Conclusion

Integrating local geometric awareness via EdgeConv significantly improves the robustness of 3D object detection. By addressing the segmentation bottleneck, the model produces more accurate and stable bounding boxes, particularly in dense or occluded environments. Future work will focus on reducing the computational cost of KNN searches and decreasing dependency on initial 2D detection performance.

Summary updated

Abstract

In this paper, we proposed a novel 3D deep learning model for object localization and object bounding boxes estimation. To increase the detection efficiency of small objects in the large scale scenes, the local neighbourhood geometric structure information of objects has been taken into the Edgeconv model, which can operate the original point clouds. We evaluated the 3D bounding box with high resolution in the RGB-D dataset and acquired stable effectiveness even under the sparse points and the strong occlusion. The experimental results indicate that our method achieved the higher mean average precision and better IOU of bounding boxes in SUN RGB-D dataset and KITTI benchmark.
BibTeX
@inproceedings{lin2019using,
  author = {Weiwei Lin and Y. Chen and Chunjin Wang and Jonathan Li},
  title = {{USING EDGECONV TO IMPROVE 3D OBJECT DETECTION FROM RGB-D DATA}},
  booktitle = {˜The},
  year = {2019},
  doi = {10.5194/isprs-archives-xlii-2-w13-835-2019},
}
Figure