Abstract
Sparse 3D Convolutional Neural Networks (CNNs) are the standard for efficient point cloud processing, yet they face challenges in fine-grained tasks that require point-wise features. Because sparse grids omit "empty" space, trilinear interpolation often lacks the necessary neighboring voxels. We introduce interpolation-aware padding, a surgical technique that pads only the specific empty voxels required for interpolation. This enables the network to learn meaningful features for empty space, significantly improving performance in 3D semantic segmentation and object detection without the memory overhead of dense padding.
Introduction
In 3D deep learning, efficiency is paramount. Sparse voxel-based CNNs achieve this by performing computations only on occupied voxels. However, tasks like semantic segmentation or object detection require features at specific point locations, typically obtained via trilinear interpolation from the surrounding eight grid corners. In a sparse representation, many of these corners are empty, leading to "blocky" results from nearest-neighbor approximations or inconsistent features from zero-padding.
The Interpolation-Aware Padding Method
The core innovation is a targeted padding scheme that bridges the gap between efficiency and feature continuity. Instead of padding a full "ring" of voxels around every occupied cell—which is memory-intensive—the proposed algorithm identifies exactly which empty voxels are needed to perform mathematically complete trilinear interpolation for every input point.
The process follows these steps:
- Voxelization: Mapping the input point cloud to a sparse grid.
- Targeted Padding: For each point, the eight nearest grid corners are identified. If a corner is empty, a "padded" voxel is instantiated.
- Feature Learning: These padded voxels are initialized to zero but participate in subsequent 3D convolutions. This allows the network to propagate context from occupied regions into these "buffer" zones.
- Interpolation: Points extract features from a complete set of eight neighbors, enabling standard trilinear interpolation.
As shown in the comparison above, interpolation-aware padding (d) is significantly more efficient than 1-ring padding (b) while providing better coverage than zero padding (a).
Experimental Evaluation
The method was evaluated on three major benchmarks: PartNet (fine-grained shape segmentation), ScanNet (indoor scene segmentation and detection), and KITTI (outdoor LiDAR segmentation).
Key Results
- Accuracy Improvements: The scheme consistently outperformed baselines, improving mean Intersection over Union (mIoU) by 2.2% on PartNet, 2.0% on ScanNet, and 2.4% on KITTI.
- Detection Gains: For 3D object detection on ScanNet, the method yielded a 2.0 point improvement in [email protected].
- Memory Efficiency: On the KITTI dataset, targeted padding required only 6.4GB of VRAM, compared to 12.9GB for 1-ring padding—a 50% reduction that makes training on large scenes much more feasible.
Discussion and Insights
A key finding is that padding acts as an "information bridge." In high-resolution sparse grids, occupied voxels can become isolated. Padding creates pathways for features to flow across small gaps during convolution. Furthermore, the network learns to populate these padded voxels with boundary-aware features rather than leaving them at zero, providing much richer context for interpolation than a hardcoded zero.
Conclusion
Interpolation-aware padding offers a simple yet effective "plug-in" for sparse CNN architectures. By strategically adding only the necessary voxels for interpolation, it allows models to achieve higher accuracy at coarser resolutions, optimizing the trade-off between computational cost and performance.
Summary updated