Automotive Radar Processing With Spiking Neural Networks: Concepts and Challenges

Bernhard Vogginger, Felix Kreutz, Javier López-Randulfe, Chen Liu, Robin Dietrich, Hector A. Gonzalez, Daniel Scholz, Nico Reeb, Daniel Auge, Julian Hille, Muhammad Arsalan, Florian Mirus, Cyprian Grassmann, Alois Knoll, Christian MayrFrontiers in Neuroscience 2022 · DOI

TL;DR

This paper explores the potential of Spiking Neural Networks (SNNs) to revolutionize automotive radar processing, aiming for significant energy efficiency gains. The authors demonstrate that SNNs can achieve competitive performance for radar target detection and classification, while drastically reducing computational cost compared to traditional methods.

What Problem Does This Paper Address?

The increasing complexity of automated driving systems necessitates sophisticated sensor processing, particularly for radar, which is robust in various conditions. However, achieving higher angular resolution with more radar components (transmitters and receivers) leads to a significant increase in the computational load for digital signal processing (DSP). This high energy consumption is a major challenge, especially for electric vehicles where it impacts range. The paper addresses this by investigating Spiking Neural Networks (SNNs), a brain-inspired computing paradigm, as a more energy-efficient alternative for radar processing.

Methods

The authors provide a comprehensive analysis of conventional automotive radar processing steps and propose how SNNs could replace or complement them. They focus on two key areas:

  • Radar Target Detection: They implement SNNs for the Constant False Alarm Rate (CFAR) algorithm, a standard method for distinguishing targets from noise. Two variants, OS-CFAR and CA-CFAR, are adapted into spiking neural networks using temporal coding. These SNNs are evaluated for their accuracy and computational efficiency.
  • Radar Target Classification: For the first time, they apply an SNN to classify detected radar objects (e.g., vehicles, pedestrians). This involves training a recurrent SNN on sequences of range-Doppler maps.

The paper also discusses the specific requirements and challenges for implementing these SNNs on neuromorphic hardware.

Datasets

The primary dataset used for evaluation in this study is the CARRADA dataset. This dataset is noted for being one of the few publicly available automotive datasets that includes radar data (range-Doppler and range-angle maps), alongside vision data. The authors also mention using a sub-dataset derived from CARRADA for target classification, focusing on sequences of regions of interest (ROIs) around detected objects.

Key Insights and Ideas

  • SNNs as a Viable Alternative: The research strongly suggests that SNNs are not just a theoretical possibility but a practical approach for automotive radar processing, offering a path towards significant energy savings.
  • Temporal Coding Efficiency: The use of temporal coding in SNNs for radar processing, particularly for CFAR detection, proves to be effective and computationally efficient, especially when compared to traditional rate-based coding.
  • Neuromorphic Hardware Synergy: The paper highlights that the true potential of SNNs for radar processing will be unlocked when implemented on dedicated neuromorphic hardware, which is designed to efficiently handle event-based, sparse computations.
  • Bridging the Gap: While traditional radar processing relies on complex DSP, SNNs offer a more biologically plausible and potentially more efficient way to extract information, especially for spatio-temporal data.
  • Classification Breakthrough: The successful application of SNNs for target classification, achieving accuracy comparable to ANNs with a fraction of the computational cost, is a significant step forward.

Key Results

  • Target Detection: The spiking OS-CFAR and CA-CFAR algorithms achieve performance close to 99% sensitivity and precision when simulated with an adequate number of time steps (around 800 for CA-CFAR, and fewer with logarithmic scaling for OS-CFAR). While requiring more operations than conventional CFAR on standard processors, the authors argue for their efficiency on neuromorphic hardware.
  • Target Classification: An SNN for target classification achieved an accuracy of 92.6% with real-valued inputs (and 90.0% with spike inputs), which is close to the 94.7% accuracy of a reference ANN. Crucially, the SNN with real-valued inputs required only 18% of the operations of the best-performing ANN, and the spike-input SNN used less than 0.5% of the operations.

Limitations and Future Work

  • Neuromorphic Hardware Constraints: The performance of spiking CFAR algorithms can be sensitive to the number of simulation time steps, which is a limitation on current digital neuromorphic hardware. The paper suggests that further research is needed to optimize these SNNs for hardware constraints.
  • Real-time Performance: While the SNNs show promise, the paper acknowledges that further work is needed to determine if neuromorphic hardware can truly outperform DSPs in terms of energy and latency for early processing steps like Fourier Transforms and CFAR.
  • Full Pipeline Integration: The current study focuses on individual processing steps. A significant future direction is to develop a complete radar processing chain composed entirely of SNNs or hybrid SNN-DSP systems.
  • Dataset Size and Diversity: The CARRADA dataset, while valuable, is noted as being limited in size and complexity. Future work could benefit from larger and more diverse datasets for more robust training and evaluation.
  • Analog/Mixed-Signal Hardware: The paper suggests that while digital neuromorphic systems are more suitable for automotive integration due to reliability concerns, analog or mixed-signal approaches might be explored for specific radar systems that don't require precise phase information.
  • Sensor Fusion: Integrating SNNs for processing data from multiple sensors (radar, camera, LiDAR) is identified as a crucial future research area for achieving comprehensive automotive perception.

Summary updated

Abstract

Frequency-modulated continuous wave radar sensors play an essential role for assisted and autonomous driving as they are robust under all weather and light conditions. However, the rising number of transmitters and receivers for obtaining a higher angular resolution increases the cost for digital signal processing. One promising approach for energy-efficient signal processing is the usage of brain-inspired spiking neural networks (SNNs) implemented on neuromorphic hardware. In this article we perform a step-by-step analysis of automotive radar processing and argue how spiking neural networks could replace or complement the conventional processing. We provide SNN examples for two processing steps and evaluate their accuracy and computational efficiency. For radar target detection, an SNN with temporal coding is competitive to the conventional approach at a low compute overhead. Instead, our SNN for target classification achieves an accuracy close to a reference artificial neural network while requiring 200 times less operations. Finally, we discuss the specific requirements and challenges for SNN-based radar processing on neuromorphic hardware. This study proves the general applicability of SNNs for automotive radar processing and sustains the prospect of energy-efficient realizations in automated vehicles.
BibTeX
@inproceedings{vogginger2022automotive,
  author = {Bernhard Vogginger and Felix Kreutz and Javier López-Randulfe and Chen Liu and Robin Dietrich and Hector A. Gonzalez and Daniel Scholz and Nico Reeb and Daniel Auge and Julian Hille and Muhammad Arsalan and Florian Mirus and Cyprian Grassmann and Alois Knoll and Christian Mayr},
  title = {{Automotive Radar Processing With Spiking Neural Networks: Concepts and Challenges}},
  booktitle = {Frontiers in Neuroscience},
  year = {2022},
  doi = {10.3389/fnins.2022.851774},
}
Figure