Abstract
Deep point cloud analytics are essential for 3D perception tasks in autonomous systems, yet they suffer from significant performance bottlenecks due to memory irregularities. Unlike 2D images, 3D point clouds are scattered irregularly in space, requiring complex tree traversals that lead to DRAM inefficiency and SRAM bank conflicts. This paper presents Crescent, an algorithm-hardware co-design system that accelerates 3D point cloud processing by intentionally introducing hardware approximations and compensating for them through software. By utilizing a two-level tree structure for neighbor searches and a novel Bank Conflict Elision (BCE) mechanism, Crescent achieves an average 1.9x speedup and 1.5x energy reduction with less than 1% accuracy loss across industry-standard benchmarks.
1. Introduction
While convolutional neural networks (CNNs) for 2D images benefit from regular, grid-like data structures, 3D point clouds are fundamentally different. Points are distributed non-uniformly in 3D space, meaning that identifying a point's neighbors—a core operation in point-based networks—requires complex tree traversals (typically using K-d trees).
In traditional hardware like GPUs or standard AI accelerators, these traversals create two primary bottlenecks:
- DRAM Inefficiency: Memory accesses are irregular and input-dependent. Hardware spends significant cycles on "pointer chasing," leading to pipeline stalls and wasted bandwidth.
- SRAM Bank Conflicts: When multiple processing elements (PEs) attempt to access the same on-chip memory bank simultaneously, the system must serialize these requests, causing further delays.
Crescent addresses these challenges by "taming" these irregularities through a strategy of controlled approximation and specialized training.
2. Methodology
Crescent employs a two-pronged approach involving hardware-algorithm co-design and software-based compensation.
2.1 Hardware/Algorithm Co-Design
To mitigate memory bottlenecks, Crescent introduces two key hardware modifications:
- Approximate Neighbor Search: Crescent partitions the K-d tree into a "top-tree" and multiple "sub-trees." By limiting backtracking to within a single sub-tree, the system can load an entire sub-tree into on-chip memory once and process it locally. This converts unpredictable DRAM accesses into efficient, predictable streaming accesses, enabling "perfect prefetching."
- Selective Bank Conflict Elision (BCE): In standard architectures, bank conflicts necessitate serialization. Crescent instead implements a "don't wait" policy. For feature computation, it reuses data from a successful request for the conflicted one. For neighbor searches, it simply skips the conflicted node traversal. This radical approach prioritizes throughput over exactness.
2.2 Software Compensation: Approximation-Aware Training
To maintain high accuracy despite hardware approximations, Crescent introduces Approximation-Aware Training. During the forward pass of the training phase, the system simulates hardware approximations, such as skipped nodes and bank conflicts. This allows the neural network's weights to learn to compensate for missing or replicated data, effectively making the model robust to the specific "flaws" of the underlying hardware.
3. Evaluation
3.1 Experimental Setup
The researchers evaluated Crescent using three industry-standard benchmarks:
- ModelNet40: For 3D object classification.
- ShapeNet: For part segmentation.
- KITTI: For real-world 3D object detection in autonomous driving scenarios.
Crescent was compared against mobile GPUs and optimized point cloud accelerators such as Mesorasi.
3.2 Key Results
- Performance: Crescent achieved an average 1.9x speedup, reaching up to 3.1x on specific models like DensePoint.
- Energy Efficiency: The system reduced energy consumption by 1.5x on average, with gains as high as 4.2x in some configurations.
- Accuracy: Despite aggressive approximations, the approximation-aware training kept accuracy loss below 1% across all tested models.
- Memory Traffic: The system successfully reduced DRAM traffic by over 40% and avoided more than 45% of SRAM bank conflicts.
4. Discussion and Limitations
Crescent demonstrates that deep learning models are inherently error-tolerant, a property that can be exploited to build "fast-but-flawed" hardware. However, this approach comes with trade-offs. The specialized training process increases training time by approximately 38% due to the overhead of simulating hardware conflicts in software.
Additionally, while Crescent excels at inference, the cost of building K-d trees for real-time sensor streams remains a separate challenge. Future work may explore scaling these sub-tree methods to massive environments, such as entire city blocks, which might require more complex recursive splitting to fit within on-chip SRAM limits.
5. Conclusion
Crescent represents a significant step forward in 3D point cloud acceleration. By embracing hardware imperfection and using software to bridge the gap, it provides a scalable solution to the memory irregularities that have long hindered 3D deep learning performance. The system proves that with the right co-design, it is possible to achieve near-perfect accuracy with significantly improved throughput and energy efficiency.
Summary updated