Cooperative Sensing and Heterogeneous Information Fusion in VCPS: A Multi-agent Deep Reinforcement Learning Approach

Xincao Xu, Kai Liu, Penglin Dai, Ruitao Xie, Jiangtao LuoIEEE Trans. ITS 2022 · OpenReview

TL;DR

This paper introduces a novel metric, "Age of View" (AoV), to quantify the quality of information fusion in vehicular cyber-physical systems (VCPSs). They propose a multi-agent deep reinforcement learning approach to optimize cooperative sensing and information uploading, aiming to maximize VCPS quality by intelligently managing sensing frequencies, data priorities, and V2I bandwidth allocation.

What Problem Does This Paper Address?

Vehicular cyber-physical systems (VCPSs) rely on collecting and fusing diverse information from vehicles to enable intelligent transportation systems (ITSs). However, this process faces significant challenges:

  • Dynamic Environment: Physical information (vehicle locations, traffic light status) changes rapidly.
  • Heterogeneous Data: Information comes from various sensors with different update rates and modalities.
  • Communication Constraints: Vehicle-to-infrastructure (V2I) communication is prone to intermittent connections and limited bandwidth.
  • Quality Measurement: Existing methods struggle to quantitatively assess the quality of fused information, especially when it's used to construct "logical views" for specific applications.

The authors aim to bridge this gap by developing a metric to measure the quality of these logical views and proposing a system to optimize their creation under realistic constraints.

Methods

The core of the proposed solution involves a multi-agent deep reinforcement learning (DRL) framework:

  • Age of View (AoV) Metric: A novel metric is introduced to quantify the quality of a logical view. AoV synthesizes three key aspects:
  • Timeliness: How fresh the information is.
  • Completeness: Whether all required information is present.
  • Consistency: How well different pieces of information align. AoV is a weighted average of normalized versions of these three factors, with lower AoV indicating higher quality.

  • Cooperative Sensing Model: A model based on a multi-class M/G/1 priority queue is used to represent information arrival and queuing at vehicles.

  • Multi-Agent DRL Approach:
    • Agents: Vehicles act as independent agents, each deciding its sensing frequencies and uploading priorities for different pieces of information.
  • Edge Node: The edge node acts as another agent, allocating V2I bandwidth to vehicles.
  • System State: Includes vehicle-sensed information, edge-cached data, and view requirements.
  • Action Space: Vehicles choose sensing frequencies and uploading priorities; edge nodes allocate V2I bandwidth.
  • Difference Reward (DR) Credit Assignment: A key innovation is how the system reward (overall VCPS quality) is distributed. Instead of a global reward, a "difference reward" is calculated for each vehicle, representing its contribution to the system's quality improvement. This helps in accurately evaluating individual agent actions.
  • V2I Bandwidth Allocation (VBA): The edge node uses predicted vehicle trajectories and view requirements to allocate bandwidth, prioritizing vehicles that can contribute most effectively.

Datasets

The authors utilized real-world vehicle trajectories collected from the Didi GAIA open dataset. They examined three specific traffic scenarios:

  1. A 3 km × 3 km area in Chengdu during rush hour on a weekday.
  2. The same area in Chengdu during late night.
  3. A 3 km × 3 km area in Xian during rush hour on a weekend.

These datasets provided detailed traffic characteristics such as vehicle traces, dwell times, speeds, and densities, allowing for realistic simulation and evaluation.

Key Insights and Ideas

  • Quantifying "View Quality": The AoV metric is a significant contribution, providing a concrete way to measure the effectiveness of information fusion in VCPS, moving beyond simple data item quality.
  • Synergy of Sensing and Fusion: The paper highlights that optimizing sensing and fusion isn't just about individual components but their synergistic interplay.
  • Importance of Credit Assignment: The "difference reward" mechanism is crucial for effective multi-agent learning in this context, allowing agents to learn from their specific contributions rather than just a global outcome.
  • Edge Intelligence for Bandwidth: Leveraging predicted vehicle trajectories and view needs at the edge to dynamically allocate V2I bandwidth is a practical approach to managing limited resources.
  • Trade-offs in Information Quality: The results demonstrate that factors like timeliness, completeness, and consistency are interconnected and need to be balanced based on application requirements. For instance, in some scenarios, low timeliness and consistency might be acceptable if completeness is high.

Key Results

The proposed multi-agent DRL approach, termed MDRAC-VBA, demonstrated superior performance across various metrics:

  • VCPS Quality: The proposed solution achieved the highest VCPS quality, outperforming RA, C-DDPG, MAC, and MAC-VBA by significant margins (e.g., 61.8% over RA, 23.8% over C-DDPG).
  • Convergence Speed: The algorithm converged much faster than other DRL-based methods (e.g., 6.8x faster than C-DDPG), attributed to a smaller action space and efficient credit assignment.
  • Average AoV: The proposed solution consistently achieved the lowest average Age of View, indicating higher quality logical views.
  • Cumulative Reward (CR): The proposed solution achieved the highest cumulative reward, reflecting its overall effectiveness.
  • Average Queuing Time (AQT) and Service Ratio (SR): The proposed solution maintained the lowest AQT and the highest SR, demonstrating efficient data handling and meeting completeness requirements.
  • Robustness: Performance was evaluated across different traffic scenarios, V2I bandwidths, and view requirements, showing consistent superiority.

Limitations and Future Work

  • Single Edge Node Focus: The current work focuses on a single edge node's coverage. Future work will investigate cooperation among multiple edge nodes to extend the system's reach and support more complex ITS applications.
  • Transmission Power Control: The paper did not explicitly consider transmission power control. Future research will explore its integration to further optimize overall system performance and power consumption.
  • Generalization: While tested on real-world data, further exploration of generalization across a wider range of scenarios and network conditions would be beneficial.

Summary updated

Abstract

Cooperative sensing and heterogeneous information fusion are critical to realize vehicular cyber-physical systems (VCPSs). This paper makes the first attempt to quantitatively measure the quality of VCPS by designing a new metric called Age of View (AoV). Specifically, we first present the system architecture where heterogeneous information can be cooperatively sensed and uploaded via vehicle-to-infrastructure (V2I) communications in vehicular edge computing (VEC). Logical views are constructed by fusing the heterogeneous information at edge nodes. Further, we formulate the problem by deriving a cooperative sensing model based on the multi-class M/G/1 priority queue, and defining the AoV by modeling the timeliness, completeness and consistency of the logical views. On this basis, a multi-agent deep reinforcement learning solution is proposed. In particular, the system state includes vehicle sensed information, edge cached information and view requirements. The vehicle action space consists of the sensing frequencies and uploading priorities of information. A difference-reward-based credit assignment is designed to divide the system reward, which is defined as the VCPS quality, into the difference reward for vehicles. Edge node allocates V2I bandwidth to vehicles based on predicted vehicle trajectories and view requirements. Finally, we build the simulation model and give a comprehensive performance evaluation, which conclusively demonstrates the superiority of the proposed solution.
BibTeX
@article{xu2022cooperative,
  author = {Xincao Xu and Kai Liu and Penglin Dai and Ruitao Xie and Jiangtao Luo},
  title = {{Cooperative Sensing and Heterogeneous Information Fusion in VCPS: A Multi-agent Deep Reinforcement Learning Approach}},
  journal = {IEEE Trans. Intell. Transp. Syst.},
  year = {2022},
  url = {https://openreview.net/forum?id=Hh1HwB9avw},
}
Figure