AI-enabled mm-Waveform Configuration for Autonomous Vehicles with Integrated Communication and Sensing

Nam H. Chu, Diep N. Nguyen, Dinh Thai Hoang, Quoc‐Viet Pham, Khoa T. Phan, Won‐Joo Hwang, Eryk Dutkiewicz

TL;DR

This paper introduces i-ICS, an AI-driven framework that allows autonomous vehicles to balance high-speed data transmission with precise radar sensing using a single millimeter-wave (mmWave) signal. By employing advanced Deep Reinforcement Learning, the system adaptively reconfigures its waveform in real-time to navigate the "tug-of-war" between communication efficiency and sensing accuracy in unpredictable traffic environments.

What Problem Does This Paper Address?

Autonomous Vehicles (AVs) traditionally use separate hardware for sensing (like radar) and communication (V2X). This is expensive, bulky, and wastes precious radio spectrum. Integrated Communications and Sensing (ICS) aims to solve this by using one signal for both tasks.

However, there is a fundamental conflict:

  1. Sensing (radar) works best with many short frames because it relies on the "preambles" (headers) of data packets to act as radar pulses. More preambles mean better velocity estimation.
  2. Communication works best with fewer, longer frames to minimize overhead and maximize the actual data being sent.

In a fast-moving car, the wireless channel quality and the amount of data needing to be sent change every millisecond. Static configurations can't keep up, leading to either "blind" radar or dropped data packets.

Methods

The authors model the waveform configuration as a Markov Decision Process (MDP) and solve it using a specialized Deep Reinforcement Learning (DRL) algorithm they call i-ICS.

Key technical components include:

  • IEEE 802.11ad Standard: They build upon the existing 60 GHz mmWave Wi-Fi standard, specifically the Single Carrier Physical Layer (SC-PHY).
  • Dueling Double Deep Q-Network (D3QN): To handle the complex, high-dimensional state of a moving car, they combine three advanced DRL techniques:
  • Deep Q-Learning: Uses neural networks to estimate the value of different waveform configurations.
  • Double Q-Learning: Prevents the AI from being "over-optimistic" about certain actions, which stabilizes learning.
  • Dueling Architecture: Separates the value of being in a certain state (e.g., "the data queue is full") from the advantage of a specific action (e.g., "send 10 frames"). This helps the AI learn which factors are actually driving performance.

Datasets

The paper relies on extensive numerical simulations rather than a static external dataset. The environment is modeled using:

  • Poisson Distribution: To simulate realistic, bursty data arrival rates at the vehicle's transmitter.
  • IEEE 802.11ad Parameters: Using real-world specs like a 60 GHz carrier frequency and 1.76 GHz sampling rate.
  • Channel Models: Three levels of Packet Error Ratio (PER) to simulate poor, normal, and strong wireless signal conditions.

Key Insights and Ideas

  • The Adaptive "Sweet Spot": The AI learns that when data traffic is low, it can afford to send many small frames to maximize radar precision. When the data queue starts to pile up, it automatically switches to longer frames to clear the buffer, even if it slightly reduces sensing accuracy.
  • Model-Free Intelligence: Unlike traditional optimization methods that require perfect knowledge of the environment (which is impossible on a highway), the DRL approach learns the optimal strategy simply through "trial and error" interactions with the simulated environment.
  • Stability via Dueling: The authors found that the dueling architecture is particularly effective for ICS because the "state" (how much data is waiting) often matters more than the specific "action" (the exact number of frames), allowing the agent to converge on a solution much faster than standard Q-learning.

Key Results

  • Performance Boost: The i-ICS approach improved joint communication and sensing performance by up to 46.26% compared to baseline methods (like greedy or fixed-frame policies).
  • Superior Learning: Compared to standard Q-learning, i-ICS achieved a 40% higher average reward and converged to a stable policy significantly faster.
  • Robustness: The system maintained low packet drop rates and high velocity-estimation accuracy across a wide range of data arrival rates and channel conditions, proving it can handle the "chaos" of real-world driving.

Limitations and Future Work

  • Simulation vs. Reality: While the simulations are rigorous, the authors acknowledge that real-world mmWave signals are highly sensitive to "blockage" (e.g., a truck passing by). Future work needs to test this on physical hardware in real traffic.
  • Single-Target Focus: The current model focuses primarily on sensing one main target vehicle. Real-world scenarios involve tracking dozens of objects simultaneously, which would significantly increase the computational complexity.
  • Fixed CPI: The Coherent Processing Interval (the window of time the AI looks at) is currently fixed. Allowing the AI to also adjust the length of this window could provide even more flexibility.

Summary updated

Abstract

Integrated Communications and Sensing (ICS) has recently emerged as an enabling technology for ubiquitous sensing and IoT applications. For ICS application to Autonomous Vehicles (AVs), optimizing the waveform structure is one of the most challenging tasks due to strong influences between sensing and data communication functions. Specifically, the preamble of a data communication frame is typically leveraged for the sensing function. As such, the higher number of preambles in a Coherent Processing Interval (CPI) is, the greater sensing task's performance is. In contrast, communication efficiency is inversely proportional to the number of preambles. Moreover, surrounding radio environments are usually dynamic with high uncertainties due to their high mobility, making the ICS's waveform optimization problem even more challenging. To that end, this paper develops a novel ICS framework established on the Markov decision process and recent advanced techniques in deep reinforcement learning. By doing so, without requiring complete knowledge of the surrounding environment in advance, the ICS-AV can adaptively optimize its waveform structure (i.e., number of frames in the CPI) to maximize sensing and data communication performance under the surrounding environment's dynamic and uncertainty. Extensive simulations show that our proposed approach can improve the joint communication and sensing performance up to 46.26% compared with other baseline methods.
BibTeX
@article{chu2023ai,
  author = {Nam H. Chu and Diep N. Nguyen and Dinh Thai Hoang and Quoc‐Viet Pham and Khoa T. Phan and Won‐Joo Hwang and Eryk Dutkiewicz},
  title = {{AI-enabled mm-Waveform Configuration for Autonomous Vehicles with Integrated Communication and Sensing}},
  journal = {IEEE Internet of Things Journal},
  year = {2023},
  doi = {10.1109/jiot.2023.3270420},
}
Figure