Introduction
Autonomous vehicles (AVs) rely on a diverse array of sensors—including cameras, LiDAR, and radar—to perceive their environment. While multimodal fusion enhances perception, individual sensors are prone to specific failures: cameras struggle in low-light or adverse weather, and LiDAR can be compromised by atmospheric interference. To ensure safety, AVs must not only detect objects but also quantify the reliability of their perceptions. HyperDUM is a novel framework that utilizes Hyperdimensional Computing (HDC) to quantify "epistemic" uncertainty—the model's lack of knowledge—directly at the feature-fusion level.
The Challenge of Real-Time Uncertainty
Existing Uncertainty Quantification (UQ) methods, such as Bayesian Neural Networks or Deep Ensembles, are often computationally prohibitive for high-speed driving due to the need for multiple inference passes. Furthermore, many current approaches calculate uncertainty only at the final output stage. This creates a "fusion bottleneck" where corrupted sensor data propagates through the network before the system identifies an error.
The HyperDUM Framework
HyperDUM addresses these issues by being a Deterministic Uncertainty Method (DUM) that operates in a single forward pass. It projects latent features from various sensors into a high-dimensional space where it learns "prototypes"—mathematical representations of known, reliable scenarios. Uncertainty is determined by the distance between incoming data and these prototypes.
To handle the complexities of spatial and spectral data, HyperDUM introduces two key mechanisms:
- Channel-wise Projection & Bundling (CPB): This treats feature channels independently, recognizing that different channels may capture environmental aspects with varying reliability.
- Patch-wise Projection & Bundling (PPB): This divides feature maps into spatial patches, allowing the model to identify localized sensor failures (e.g., a smudge on a lens) without discarding the entire frame.
These quantified uncertainties act as weights during fusion, effectively "turning down the volume" on unreliable sensor inputs before they can degrade the final perception output.
Experimental Setup and Robustness
The framework was evaluated on two major multimodal datasets: aiMotive (for 3D object detection) and DeLiVER (for semantic segmentation). To test the system's limits, the researchers introduced synthetic "corner cases" representing sensor degradation, such as motion blur and LiDAR foggification.
Key Results
HyperDUM demonstrates significant improvements in both performance and efficiency:
- Perception Accuracy: The model achieved a 2.01% increase in Average Precision (AP) for 3D detection on aiMotive and a 1.29% improvement in mean Intersection over Union (mIoU) for segmentation on DeLiVER.
- Computational Efficiency: HyperDUM requires 2.36x fewer FLOPs and up to 38.30x fewer parameters than previous state-of-the-art deterministic methods like LDU.
- Resilience: Under extreme conditions like heavy fog or motion blur, performance degradation was limited to just 0.51%, significantly outperforming baseline models.
- Calibration: The model achieved the lowest Expected Calibration Error (ECE), ensuring that its confidence scores accurately reflect its real-world performance.
Conclusion
HyperDUM provides a scalable and efficient solution for uncertainty-aware multimodal fusion in autonomous systems. By quantifying uncertainty at a granular level before fusion, it prevents the propagation of sensor errors. Future work will focus on unsupervised prototype learning and hardware-level optimizations for specialized AI accelerators.
Summary updated