TL;DR
HyperLiDAR introduces a lightweight framework for on-device, post-deployment LiDAR segmentation using Hyperdimensional Computing (HDC). By replacing costly gradient descent with simple vector arithmetic and implementing a smart 5% buffer selection strategy, it achieves up to a 13.8× speedup in retraining while maintaining or exceeding state-of-the-art accuracy under real-world domain shifts.
What Problem Does This Paper Address?
LiDAR semantic segmentation is crucial for autonomous vehicles to understand 3D environments by assigning labels (like "car" or "pedestrian") to millions of individual points. However, when a vehicle moves into a new environment—such as transitioning from a sunny urban center to a foggy rural road—the distribution of the data shifts, causing model accuracy to plummet.
To fix this, models must adapt post-deployment by retraining on new, streaming data. Doing this on edge devices presents two massive hurdles:
- Resource Constraints: Conventional deep neural networks (DNNs) require massive computational power and memory to train, making on-device backpropagation impractical for edge hardware.
- Catastrophic Forgetting: Standard DNNs tend to overwrite previously learned knowledge when exposed to new environments.
While existing research tackles cross-domain adaptation, these solutions remain too heavy for resource-constrained edge systems.
Methods
HyperLiDAR shifts the paradigm by utilizing Hyperdimensional Computing (HDC), a brain-inspired computing model that operates on high-dimensional, low-precision vectors (hypervectors) using simple, highly parallelizable element-wise additions and multiplications.
The framework operates in three stages:
- Cloud Pre-training: A projection-based feature extractor is trained offline on large-scale data to convert 3D point clouds into efficient 2D image representations. This feature extractor is frozen before deployment.
- Post-Deployment Adaptation: On the edge device, features are mapped into high-dimensional hypervectors. Instead of calculating gradients, HyperLiDAR updates class hypervectors using basic vector addition for correct classes and subtraction for mispredicted classes.
- Buffer Selection Strategy: Processing 50k–60k points per scan is a major bottleneck. HyperLiDAR introduces a coreset selection strategy that updates the model using only 5% of the data. This buffer is split equally between Hard Samples (points with the highest classification loss from the previous pass) and Random Samples (to preserve generalization).
Datasets
The authors evaluated HyperLiDAR on two widely recognized autonomous driving benchmarks:
- SemanticKITTI: Comprising 43,551 scans from 22 driving sequences. Eight scenes were used for pre-training, two for post-deployment adaptation, and one completely unseen scene for final evaluation.
- nuScenes: Featuring 1,000 urban driving scenes across diverse weather conditions. The dataset was split 8:2 (560 scenes for pre-training, 140 for post-deployment), leaving 150 unseen scenes for evaluation.
Key Insights and Ideas
- Gradient-Free Learning at the Edge: Bypassing backpropagation entirely allows HyperLiDAR to adapt to new environments without the massive power budgets typically demanded by deep learning.
- The Efficiency of "Less is More": Retraining on just 5% of the most informative data points yields virtually identical accuracy to using 100% of the data, effectively breaking the data-volume bottleneck inherent to dense LiDAR scans.
- Stable Optimization Trajectories: Unlike DNN baselines, which suffer from volatile accuracy oscillations during online adaptation due to catastrophic forgetting, HyperLiDAR exhibits a smooth, stable, and monotonically increasing convergence curve.
Key Results
- Retraining Speedups: On an NVIDIA RTX 4090 GPU, HyperLiDAR achieved 84.50 FPS during retraining, delivering a 2.25× speedup over CENET and a 13.8× speedup over SALUDA.
- Segmentation Accuracy: On SemanticKITTI, HyperLiDAR achieved 54.0% mIoU, outperforming SALUDA (49.1%). On the larger nuScenes benchmark, it reached 61.6% mIoU, significantly outperforming CENET (54.5%).
- Hardware Deployment: When deployed on a custom HDC ASIC accelerator (FSL-HDnn), HyperLiDAR achieved a throughput of 16.3 FPS, demonstrating its readiness for specialized, low-power edge hardware.
Limitations and Future Work
While HyperLiDAR excels at efficient on-device adaptation, it currently relies on having access to ground-truth labels during the post-deployment adaptation phase. In true zero-intervention deployments, obtaining real-time labels is challenging.
The authors note that future work will focus on extending HyperLiDAR to label-scarce adaptation settings (leveraging unsupervised or self-supervised techniques) and exploring lifelong learning scenarios where the model continuously updates over extended deployment lifecycles.
Summary updated