Hyperdimensional Uncertainty Quantification for Multimodal Uncertainty Fusion in Autonomous Vehicles Perception

Luke Chen, Junyao Wang, Trier Mortlock, Pramod P. Khargonekar, Mohammad Abdullah Al FaruqueCVPR 2025 · DOI

Introduction

Autonomous vehicles (AVs) rely on a diverse array of sensors—including cameras, LiDAR, and radar—to perceive their environment. While multimodal fusion enhances perception, individual sensors are prone to specific failures: cameras struggle in low-light or adverse weather, and LiDAR can be compromised by atmospheric interference. To ensure safety, AVs must not only detect objects but also quantify the reliability of their perceptions. HyperDUM is a novel framework that utilizes Hyperdimensional Computing (HDC) to quantify "epistemic" uncertainty—the model's lack of knowledge—directly at the feature-fusion level.

The Challenge of Real-Time Uncertainty

Existing Uncertainty Quantification (UQ) methods, such as Bayesian Neural Networks or Deep Ensembles, are often computationally prohibitive for high-speed driving due to the need for multiple inference passes. Furthermore, many current approaches calculate uncertainty only at the final output stage. This creates a "fusion bottleneck" where corrupted sensor data propagates through the network before the system identifies an error.

The HyperDUM Framework

HyperDUM addresses these issues by being a Deterministic Uncertainty Method (DUM) that operates in a single forward pass. It projects latent features from various sensors into a high-dimensional space where it learns "prototypes"—mathematical representations of known, reliable scenarios. Uncertainty is determined by the distance between incoming data and these prototypes.

Overview of the HyperDUM framework, illustrating the pipeline from multi-sensor data extraction to uncertainty-aware fusion.

To handle the complexities of spatial and spectral data, HyperDUM introduces two key mechanisms:

  • Channel-wise Projection & Bundling (CPB): This treats feature channels independently, recognizing that different channels may capture environmental aspects with varying reliability.
  • Patch-wise Projection & Bundling (PPB): This divides feature maps into spatial patches, allowing the model to identify localized sensor failures (e.g., a smudge on a lens) without discarding the entire frame.

These quantified uncertainties act as weights during fusion, effectively "turning down the volume" on unreliable sensor inputs before they can degrade the final perception output.

Experimental Setup and Robustness

The framework was evaluated on two major multimodal datasets: aiMotive (for 3D object detection) and DeLiVER (for semantic segmentation). To test the system's limits, the researchers introduced synthetic "corner cases" representing sensor degradation, such as motion blur and LiDAR foggification.

Examples of synthetic corner cases applied to LiDAR (foggification) and camera (blur, exposure) data to test model robustness.

Key Results

HyperDUM demonstrates significant improvements in both performance and efficiency:

  • Perception Accuracy: The model achieved a 2.01% increase in Average Precision (AP) for 3D detection on aiMotive and a 1.29% improvement in mean Intersection over Union (mIoU) for segmentation on DeLiVER.
  • Computational Efficiency: HyperDUM requires 2.36x fewer FLOPs and up to 38.30x fewer parameters than previous state-of-the-art deterministic methods like LDU.
  • Resilience: Under extreme conditions like heavy fog or motion blur, performance degradation was limited to just 0.51%, significantly outperforming baseline models.
  • Calibration: The model achieved the lowest Expected Calibration Error (ECE), ensuring that its confidence scores accurately reflect its real-world performance.

Conclusion

HyperDUM provides a scalable and efficient solution for uncertainty-aware multimodal fusion in autonomous systems. By quantifying uncertainty at a granular level before fusion, it prevents the propagation of sensor errors. Future work will focus on unsupervised prototype learning and hardware-level optimizations for specialized AI accelerators.

Summary updated

BibTeX
@inproceedings{chen2025hyperdimensional,
  author = {Luke Chen and Junyao Wang and Trier Mortlock and Pramod P. Khargonekar and Mohammad Abdullah Al Faruque},
  title = {{Hyperdimensional Uncertainty Quantification for Multimodal Uncertainty Fusion in Autonomous Vehicles Perception}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.02078},
}
Figure