Information Entropy Guided Height-aware Histogram for Quantization-friendly Pillar Feature Encoder

Sifan Zhou, Zhihang Yuan, Dawei Yang, Xubin Wen, Xing Jun Hu, Yuguang Shi, Ziyu Zhao, Xiaobo Lu

TL;DR

This paper introduces PillarHist, a novel "height-aware histogram" module for 3D object detection. PillarHist significantly improves performance and quantization-friendliness in pillar-based detectors by better preserving height information and stabilizing feature distributions, without adding significant computational overhead.

What Problem Does This Paper Address?

3D object detection is crucial for applications like autonomous driving and robotics, demanding both high accuracy and real-time performance. Pillar-based methods offer a good balance of efficiency and accuracy due to their compact representation. However, existing pillar-based detectors suffer from two main issues in their Pillar Feature Encoding (PFE) stage:

  1. Information Loss in Height Dimension: Current methods often fail to effectively capture the detailed information along the vertical (height) axis, which is vital for distinguishing objects and their spatial relationships.
  2. Quantization Challenges: The PFE stage can produce features with large numerical distribution differences, making them susceptible to performance degradation when quantized (e.g., for deployment on resource-constrained edge devices). This is often exacerbated by operations like max-pooling, which can lose fine-grained geometric details.

Methods

The authors propose PillarHist, a novel height-aware pillar feature encoder designed to address the aforementioned limitations. Its core components are:

  • Height-Aware Histogram Encoding: Instead of relying on max-pooling, PillarHist divides each pillar into multiple bins along the height dimension. It then computes a histogram of points falling into each bin. This captures the distribution of points across different heights.
  • Intensity-Weighted Histogram: To further enrich the height information, the histogram is weighted by the LiDAR reflection intensity of each point. This provides a more nuanced representation of the pillar's vertical structure.
  • Pillar Center Coordinates: The (x, y) coordinates of the pillar's center are also included. This provides global spatial context, helping to distinguish pillars that might have similar internal point distributions but are located differently in space.
  • Information Entropy Guidance: The paper analyzes the impact of the number of histogram bins using information entropy, providing a theoretical basis for selecting an optimal number of bins.
  • Pillar-Level MLP: The computed histograms and pillar center coordinates are concatenated and then passed through a linear layer (MLP) at the pillar level, not the point level. This significantly reduces computation compared to point-wise operations.
  • Quantization-Friendly Design: By creating a more stable and constrained distribution of features, PillarHist makes the PFE input more amenable to quantization, reducing performance drops.

Crucially, PillarHist is designed as a plugin module that can be seamlessly integrated into existing pillar-based architectures without requiring complex modifications to other parts of the network.

Datasets

The paper evaluates PillarHist on two widely used autonomous driving datasets:

  • KITTI: A standard benchmark for 3D object detection, used for evaluating performance on car and pedestrian detection.
  • nuScenes: A more challenging dataset with a larger number of objects and diverse scenes, used for comprehensive evaluation of detection metrics like NDS and mAP.

Key Insights and Ideas

  • Height is King (for Pillars): The paper provides strong evidence that the height dimension is critically important for pillar-based PFE. Simply inputting height-related information (even without other features) yields comparable performance to the baseline, highlighting its underutilization in prior methods.
  • Max-Pooling is a Bottleneck: The visualization of max-pooling weights clearly shows how it discards significant geometric information, especially from lower points within a pillar, leading to a loss of detail.
  • Histograms as a Superior Alternative: PillarHist demonstrates that histograms can effectively capture height distribution and geometric information without the information loss associated with max-pooling.
  • Quantization Benefits from Stable Features: The paper shows that by creating more stable and predictable feature distributions, PillarHist significantly mitigates the performance degradation typically seen after quantizing PFE outputs.
  • Plugin Architecture is Key: The design of PillarHist as a drop-in replacement for the PFE module makes it highly practical and easy to adopt by the research community.

Key Results

  • Performance Gains: PillarHist consistently improves performance across various baseline models. For example, it achieves an average improvement of 1.5% NDS on nuScenes and 1.43 mAP for the vehicle class on KITTI.
  • Quantization Improvement: When models are quantized to INT8, those incorporating PillarHist experience significantly smaller performance drops (e.g., 1.6 NDS drop for PH-PillarNet vs. 6.8 NDS for PillarNet) and often perform comparably to their unquantized counterparts without PillarHist.
  • Efficiency: PillarHist operates at the pillar level, significantly reducing computational cost. It achieves this while using only about 40% of the computational cost (GFLOPs) compared to some other PFE methods.
  • Generalization: The module shows strong generalization capabilities, improving performance on different pillar-based architectures (PointPillars, CenterPoint-Pillar, PillarNet) and even showing some benefits when integrated into a voxel-based detector (SECOND).

Limitations and Future Work

  • Voxel-based Integration: While PillarHist shows some improvement when integrated into a voxel-based detector (SECOND), the gains are smaller. The authors attribute this to voxel-based methods already encoding height information more explicitly. Further exploration of how to best leverage PillarHist's histogram approach in voxel-based architectures could be a future direction.
  • Bin Number Optimization: The paper performs an ablation study on the number of bins, finding a sweet spot around 64. However, the optimal number might vary across datasets or specific tasks, suggesting that adaptive bin selection could be an area for future research.
  • Further Quantization Exploration: While PillarHist makes PFE quantization-friendly, further research into optimal quantization strategies for the entire network, especially with PillarHist integrated, could yield even greater compression benefits.

Summary updated

Abstract

Real-time and high-performance 3D object detection plays a critical role in autonomous driving and robotics. Recent pillar-based 3D object detectors have gained significant attention due to their compact representation and low computational overhead, making them suitable for onboard deployment and quantization. However, existing pillar-based detectors still suffer from information loss along height dimension and large numerical distribution difference during pillar feature encoding (PFE), which severely limits their performance and quantization potential. To address above issue, we first unveil the importance of different input information during PFE and identify the height dimension as a key factor in enhancing 3D detection performance. Motivated by this observation, we propose a height-aware pillar feature encoder, called PillarHist. Specifically, PillarHist statistics the discrete distribution of points at different heights within one pillar with the information entropy guidance. This simple yet effective design greatly preserves the information along the height dimension while significantly reducing the computation overhead of the PFE. Meanwhile, PillarHist also constrains the arithmetic distribution of PFE input to a stable range, making it quantization-friendly. Notably, PillarHist operates exclusively within the PFE stage to enhance performance, enabling seamless integration into existing pillar-based methods without introducing complex operations. Extensive experiments show the effectiveness of PillarHist in terms of both efficiency and performance.
BibTeX
@misc{zhou2024information,
  author = {Sifan Zhou and Zhihang Yuan and Dawei Yang and Xubin Wen and Xing Jun Hu and Yuguang Shi and Ziyu Zhao and Xiaobo Lu},
  title = {{Information Entropy Guided Height-aware Histogram for Quantization-friendly Pillar Feature Encoder}},
  year = {2024},
  eprint = {2405.18734},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2405.18734},
}
Figure