PDM-SSD: Single-Stage Three-Dimensional Object Detector With Point Dilation

Ao Liang, Haiyang Hua, Jian Fang, Wenyu Chen, Huaici Zhao

Abstract

Current Point-based detectors can only learn from the provided points, with limited receptive fields and insufficient global learning capabilities for such targets. In this paper, we present a novel Point Dilation Mechanism for single-stage 3D detection (PDM-SSD) that takes advantage of these two representations. Specifically, we first use a PointNet-style 3D backbone for efficient feature encoding. Then, a neck with Point Dilation Mechanism (PDM) is used to expand the feature space, which involves two key steps: point dilation and feature filling. The former expands points to a certain size grid centered around the sampled points in Euclidean space. The latter fills the unoccupied grid with feature for backpropagation using spherical harmonic coefficients and Gaussian density function in terms of direction and scale. Next, we associate multiple dilation centers and fuse coefficients to obtain sparse grid features through height compression. Finally, we design a hybrid detection head for joint learning, where on one hand, the scene heatmap is predicted to complement the voting point set for improved detection accuracy, and on the other hand, the target probability of detected boxes are calibrated through feature fusion. On the challenging Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) dataset, PDM-SSD achieves state-of-the-art results for multi-class detection among single-modal methods with an inference speed of 68 frames. We also demonstrate the advantages of PDM-SSD in detecting sparse and incomplete objects through numerous object-level instances. Additionally, PDM can serve as an auxiliary network to establish a connection between sampling points and object centers, thereby improving the accuracy of the model without sacrificing inference speed.

Introduction: The Grid vs. Point Dilemma

In the domain of 3D object detection using LiDAR, researchers typically choose between two primary representations: grid-based (voxels or pillars) and point-based (raw coordinates).

Grid-based methods are inherently robust because their convolutional layers naturally "spread" features into empty space, which helps in detecting incomplete or sparse objects. However, these methods are often computationally heavy and memory-intensive. Conversely, point-based methods are significantly faster and more memory-efficient. Their main drawback is a "discontinuous receptive field"—because they only learn from existing points, they struggle to understand the global structure of an object if the data is sparse or if parts of the object are occluded.

Comparison of (a) grid-based, (b) point-based, and (c) the proposed PDM-SSD approach.

The authors of PDM-SSD propose a solution that gives point-based detectors the "hallucination" powers of grid-based models without sacrificing their characteristic speed.

The PDM-SSD Architecture

The architecture of PDM-SSD follows a single-stage pipeline designed for both efficiency and accuracy. It consists of three main components:

  1. PointNet-style Backbone: The model utilizes a standard, efficient backbone that employs Set Abstraction (SA) modules to downsample the point cloud and extract local features.
  2. Point Dilation Mechanism (PDM) Neck: This is the core innovation. It takes sparse features from the backbone and "dilates" them into a 2D grid. To make this empty space meaningful for the neural network, the authors use two techniques for Feature Filling:
  3. Angle Coefficients: Utilizing spherical harmonics, the model learns how features should vary based on the observation angle.
  4. Scale Coefficients: A Gaussian density function models how features decay or change based on the distance from the original sampled points.
  5. Hybrid Detection Head: The model employs a "Joint Learning" approach. It predicts a scene heatmap to identify object centers even where points are missing, and combines this with a point-based voting network. These two streams are fused using a channel attention mechanism to produce final 3D bounding boxes.
The overall architecture of PDM-SSD, detailing the PDM neck and hybrid detection head.

Key Innovations: Filling the Void

The most significant insight of PDM-SSD is the realization that point-based models often fail not because they lack features, but because they lack connectivity in empty space. The PDM provides a mathematical bridge—via spherical harmonics and Gaussians—to allow features to flow into unoccupied regions.

Furthermore, the PDM module is highly flexible. It can be used as an auxiliary network during training only, which improves the backbone's feature representation without adding any overhead to inference time. Alternatively, it can be used in joint mode for maximum detection accuracy. This flexibility allows the model to address the "low probability" issue, where sparse objects might be correctly localized but assigned low confidence scores. By fusing grid-based heatmap features, PDM-SSD "calibrates" these probabilities, significantly improving recall for difficult targets.

Experimental Results

Evaluated on the KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute) object detection benchmark, PDM-SSD achieved state-of-the-art results for single-modal methods:

  • Performance: On the KITTI test set, PDM-SSD achieved 80.87% AP (Moderate) for 3D Car detection, outperforming other single-stage, point-based methods such as IA-SSD and 3DSSD.
  • Efficiency: The model runs at 68 FPS in joint mode. When utilized in auxiliary mode, it maintains the base detector's speed of 84 FPS while still providing a notable performance boost.
  • Robustness: Qualitative results demonstrate that PDM-SSD successfully detects cars that are almost entirely occluded or represented by only a handful of points, scenarios where traditional point-based models typically fail.

Conclusion

PDM-SSD effectively bridges the gap between efficient point-based methods and robust grid-based reasoning. By introducing the Point Dilation Mechanism, the model can "fill in the blanks" of sparse LiDAR data, allowing it to reason about unoccupied space and achieve high accuracy at real-time speeds. Future work will explore adaptive dilation sizes for different object classes and more complex covariance matrices to better capture 3D geometry.

Summary updated

Abstract

Current Point-based detectors can only learn from the provided points, with limited receptive fields and insufficient global learning capabilities for such targets. In this paper, we present a novel Point Dilation Mechanism for single-stage 3D detection (PDM-SSD) that takes advantage of these two representations. Specifically, we first use a PointNet-style 3D backbone for efficient feature encoding. Then, a neck with Point Dilation Mechanism (PDM) is used to expand the feature space, which involves two key steps: point dilation and feature filling. The former expands points to a certain size grid centered around the sampled points in Euclidean space. The latter fills the unoccupied grid with feature for backpropagation using spherical harmonic coefficients and Gaussian density function in terms of direction and scale. Next, we associate multiple dilation centers and fuse coefficients to obtain sparse grid features through height compression. Finally, we design a hybrid detection head for joint learning, where on one hand, the scene heatmap is predicted to complement the voting point set for improved detection accuracy, and on the other hand, the target probability of detected boxes are calibrated through feature fusion. On the challenging Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) dataset, PDM-SSD achieves state-of-the-art results for multi-class detection among single-modal methods with an inference speed of 68 frames. We also demonstrate the advantages of PDM-SSD in detecting sparse and incomplete objects through numerous object-level instances. Additionally, PDM can serve as an auxiliary network to establish a connection between sampling points and object centers, thereby improving the accuracy of the model without sacrificing inference speed. Our code will be available at https://github.com/AlanLiangC/PDM-SSD.git.
BibTeX
@misc{liang2025pdm,
  author = {Ao Liang and Haiyang Hua and Jian Fang and Wenyu Chen and Huaici Zhao},
  title = {{PDM-SSD: Single-Stage Three-Dimensional Object Detector With Point Dilation}},
  year = {2025},
  eprint = {2502.07822},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.07822},
}
Figure