A Multi-Agent Security Testbed for the Analysis of Attacks and Defenses in Collaborative Sensor Fusion

R. Spencer Hallyburton, David Hunt, Shaocheng Luo, Miroslav PajicarXiv 2024 · DOI · arXiv

Abstract

Collaborative autonomous vehicle (AV) systems leverage multi-sensor/multi-agent (MS/MA) fusion to overcome individual sensor limitations. However, this connectivity introduces vulnerabilities to cyber-attacks. We present MAST (Multi-Agent Security Testbed), an open-source ROS2-based framework that enables real-time security analysis of collaborative AV swarms. MAST bridges the AVstack library with ROS, allowing for the simulation of coordinated and uncoordinated attacks on sensor fusion pipelines. Our analysis reveals that while collaborative systems are resilient to data loss, they are highly susceptible to "ghost" object injection by adversaries, particularly when attacks are coordinated to maintain physical plausibility.

Introduction

Individual autonomous vehicles often struggle with environmental adversity, such as heavy rain or sensor occlusions. To mitigate these issues, the industry is shifting toward Multi-Sensor/Multi-Agent (MS/MA) systems, where vehicles share data via V2V and V2I communications to form a collective operating picture. However, this connectivity allows malicious data from a single compromised agent to propagate through the entire network. Existing AV development tools often lack the capability to simulate such security threats in real-time. This paper introduces MAST, a scalable testbed designed to bridge this gap by enabling the injection of adversaries into multi-agent swarms.

The MAST Framework

MAST is built on ROS2 to ensure modularity and scalability. A core innovation is the AVstack-ROS Bridge, which enables real-time, event-driven simulation of state-of-the-art AV algorithms. The framework uses a high-level configuration system to automatically assemble perception and tracking pipelines at runtime.

The MAST architecture facilitates the injection of adversary nodes between sensing and tracking modules, allowing for the study of how malicious data propagates to a centralized Command Center.

To simulate attacks, MAST employs dynamic topic remapping, allowing researchers to insert adversary nodes that intercept and modify sensor data on the fly. These agents report to a Command Center (CC), which performs collation and clustering (e.g., using DBSCAN) to create a unified view of the environment.

Experimental Setup

The testbed utilizes the CARLA infrastructure dataset, featuring multi-modal sensor data from both mobile "ego" vehicles and static infrastructure sensors. The setup typically involves four static agents providing bird's-eye views, fused with local mobile agent data. Experiments are conducted in an event-driven manner, ensuring repeatability while allowing the simulation to scale based on computational resources.

Security Analysis and Results

Our evaluation focused on the propagation of "false positives" (ghost objects) versus "false negatives" (missing data).

Vulnerability to Ghost Objects

The most significant finding is the vulnerability of multi-agent fusion to false positives. A single adversary reporting a consistent "ghost" object can often deceive the central fusion logic, as the system lacks a robust mechanism to verify the absence of an object from other perspectives. In uncoordinated attacks with three adversaries, false positives at the Command Center increased by over 23x.

Coordinated vs. Uncoordinated Attacks

In uncoordinated attacks, adversaries act independently, creating noise that is relatively easy to filter. However, in coordinated attacks, adversaries synchronize their malicious data to make ghost objects appear physically plausible. While these attacks produce fewer total false positives, they are more successful at creating persistent fake tracks that can trigger dangerous vehicle responses.

The internal Ownship Operations pipeline distinguishes between local safety-critical functions and collaborative mission-critical functions.

Safety-Critical Partitioning

The results suggest a clear architectural partition: local "safety-critical" functions, such as lane keeping or emergency braking, should rely primarily on trusted local sensors. In contrast, "mission-critical" functions, like long-range routing, can benefit from the broader (though potentially untrusted) collaborative network.

Conclusion

MAST provides a vital tool for analyzing the security of collaborative AV systems. Future work will focus on implementing "security-aware" fusion algorithms and testing the framework with larger swarms of mobile agents. By identifying the specific vulnerabilities of multi-agent pipelines, MAST paves the way for more resilient autonomous infrastructure.

Summary updated

Abstract

The performance and safety of autonomous vehicles (AVs) deteriorates under adverse environments and adversarial actors. The investment in multi-sensor, multi-agent (MSMA) AVs is meant to promote improved efficiency of travel and mitigate safety risks. Unfortunately, minimal investment has been made to develop security-aware MSMA sensor fusion pipelines leaving them vulnerable to adversaries. To advance security analysis of AVs, we develop the Multi-Agent Security Testbed, MAST, in the Robot Operating System (ROS2). Our framework is scalable for general AV scenarios and is integrated with recent multi-agent datasets. We construct the first bridge between AVstack and ROS and develop automated AV pipeline builds to enable rapid AV prototyping. We tackle the challenge of deploying variable numbers of agent/adversary nodes at launch-time with dynamic topic remapping. Using this testbed, we motivate the need for security-aware AV architectures by exposing the vulnerability of centralized multi-agent fusion pipelines to (un)coordinated adversary models in case studies and Monte Carlo analysis.
BibTeX
@misc{hallyburton2024multi,
  author = {R. Spencer Hallyburton and David Hunt and Shaocheng Luo and Miroslav Pajic},
  title = {{A Multi-Agent Security Testbed for the Analysis of Attacks and Defenses in Collaborative Sensor Fusion}},
  year = {2024},
  eprint = {2401.09387},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2401.09387},
}
Figure