Voxel Transformer with Density-Aware Deformable Attention for 3D Object Detection

Taeho Kim, Joohee KimSensors 2023 · DOI

TL;DR

This paper introduces VoTr-DADA, a novel 3D object detection backbone that enhances the Voxel Transformer (VoTr) by incorporating density-aware deformable attention. This new module adaptively focuses on important regions within point clouds, leading to improved detection accuracy, especially for distant objects, without a significant increase in computational cost.

What Problem Does This Paper Address?

Autonomous driving systems rely heavily on accurate 3D object detection from sensor data like LiDAR point clouds. While transformer-based models like VoTr excel at capturing long-range dependencies in 3D data, their fixed receptive fields can limit their ability to detect sparse or distant objects effectively. Existing methods to address this, such as deformable attention, often incur high computational costs, especially when applied to 3D data. This paper aims to bridge this gap by developing a more efficient and adaptable attention mechanism for 3D object detection.

Methods

The core innovation is the Density-Aware Deformable Attention (DADA) module. Unlike traditional deformable attention that uses lightweight CNNs to learn offsets for attention, DADA leverages voxel density information. Here's how it works:

  • Density-Guided Attention: For a given querying voxel, DADA first identifies potential attending voxels using local and dilated attention, similar to VoTr.
  • Highest-Density Voxel Selection: Crucially, DADA then uses a density function to efficiently locate the voxel with the highest point density within a defined search range around the attending voxels. This is achieved through an octree-based subdivision method.
  • Deformable Offset: The DADA module then "deforms" the attending voxels towards this highest-density voxel. This adaptive focusing allows the model to prioritize informative regions.
  • Efficient Feature Extraction: By using density information instead of learning complex offsets, DADA significantly reduces computational overhead while still achieving a flexible receptive field.

The overall architecture, VoTr-DADA, integrates this DADA module into the existing VoTr backbone, replacing some of its submanifold voxel modules.

Datasets

The proposed method was evaluated on two widely used benchmark datasets for 3D object detection:

  • KITTI Dataset: A standard dataset for autonomous driving, used for evaluating performance on cars, pedestrians, and cyclists.
  • Waymo Open Dataset (WOD): A larger and more challenging dataset, also used for vehicle detection.

Key Insights and Ideas

  • Density as a Signal for Importance: The paper highlights that voxel density is a powerful, yet underutilized, signal for guiding attention mechanisms in 3D object detection. High-density regions often correspond to more salient object features.
  • Efficiency through Density: By using density to guide deformable attention, the authors achieve improved performance without the computational burden of learning explicit offsets via CNNs. This makes the approach more practical for real-time applications.
  • Distant Object Performance: The DADA module's ability to adaptively focus on relevant areas proves particularly beneficial for detecting objects at greater distances, where point clouds are sparser.
  • Balancing Global and Local Context: The integration of DADA modules within the VoTr framework allows for a better balance between capturing broad contextual information and focusing on fine-grained details.

Key Results

  • KITTI Dataset: VoTr-DADA (TSD) achieved 84.63% mAP on the moderate car class and 78.90% mAP on the hard car class, outperforming previous methods. It also showed improvements over the vanilla VoTr-TSD on the test set.
  • Waymo Open Dataset: VoTr-DADA demonstrated significant improvements, with VoTr-DADA (SSD) and VoTr-DADA (TSD) showing gains of 1.03% and 0.74% mAP in the LEVEL 1 category, respectively. Notably, improvements were also observed for distant objects (30-50m and 50m-Inf).
  • Inference Speed: VoTr-DADA maintained computational efficiency, with a processing speed of 13.32 Hz for the single-stage detector, adding only a small latency (around 8 ms) compared to the vanilla VoTr. It was also faster than VoTr with convolution-based deformable attention.
  • Model Parameters: The DADA module led to a reduction in model parameters (0.3M) compared to using 3D convolution-based deformable attention.

Limitations and Future Work

  • Inference Latency: While efficient, the paper acknowledges that VoTr-DADA still experiences some latency.
  • Future Work: The authors plan to explore more efficient transformer-based architectures to achieve real-time performance for 3D detection. They also suggest further investigation into optimizing the DADA module for even greater efficiency.

Summary updated

Abstract

The Voxel Transformer (VoTr) is a prominent model in the field of 3D object detection, employing a transformer-based architecture to comprehend long-range voxel relationships through self-attention. However, despite its expanded receptive field, VoTr's flexibility is constrained by its predefined receptive field. In this paper, we present a Voxel Transformer with Density-Aware Deformable Attention (VoTr-DADA), a novel approach to 3D object detection. VoTr-DADA leverages density-guided deformable attention for a more adaptable receptive field. It efficiently identifies key areas in the input using density features, combining the strengths of both VoTr and Deformable Attention. We introduce the Density-Aware Deformable Attention (DADA) module, which is specifically designed to focus on these crucial areas while adaptively extracting more informative features. Experimental results on the KITTI dataset and the Waymo Open dataset show that our proposed method outperforms the baseline VoTr model in 3D object detection while maintaining a fast inference speed.
BibTeX
@inproceedings{kim2023voxel,
  author = {Taeho Kim and Joohee Kim},
  title = {{Voxel Transformer with Density-Aware Deformable Attention for 3D Object Detection}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23167217},
}
Figure