Autonomous vehicles (AVs) and mobile robots are inherently constrained by their physical line-of-sight. To overcome occlusions and extend their sensing range, modern systems leverage Vehicle-to-Vehicle (V2V) and Vehicle-to-Everything (V2X) communication to share sensor insights—a paradigm known as collaborative or cooperative perception (Hurl et al.; Zhang et al.). By fusing data from multiple vantage points, a fleet can collectively "see" around corners and anticipate hidden hazards.
However, this interconnectedness introduces a critical security vulnerability. To benefit from collective sensing, an ego agent must inherently trust external inputs. If a single participant in the network is compromised or malicious, it can broadcast fabricated data that poisons the shared environmental model (Hu et al.; Xia et al.). Traditional cybersecurity measures like network encryption are insufficient here: they secure the transmission channel, but they cannot verify the adversarial intent of the data content itself (Tu et al.).
Anatomy of an Attack: Weaponizing Shared Representations
To understand why collaborative perception is vulnerable, we must look at how data is shared. Transmitting raw, high-bandwidth camera or LiDAR streams over wireless networks introduces severe latency bottlenecks. To circumvent this, modern architectures use feature-level fusion: each vehicle processes its raw sensor data locally through a deep neural network, extracts intermediate feature maps (such as Bird's Eye View representations), and transmits these compressed embeddings to peer vehicles.
An attacker can exploit this pipeline through several distinct vectors:
1. Feature-Level Poisoning
Because the fusion layer directly integrates incoming feature maps into the ego vehicle's neural network, subtle modifications to these maps can act as adversarial perturbations. As demonstrated by Tu et al., an attacker can inject imperceptible noise into shared intermediate representations. This noise propagates through the receiving agent's network, causing a total collapse of collective perception—either erasing real obstacles or inducing hallucinations of "ghost" objects (Hu et al.; Zhang et al.).
2. Spatial Misalignment via Pose Manipulation
Before an ego vehicle can fuse an incoming feature map, it must align the sender's coordinate framework with its own. This alignment relies on spatial metadata, specifically the sender's six-degree-of-freedom pose and GPS coordinates.
Attacks like AdvGPS (Li et al.) and PosePert (Zhang et al.) exploit this dependency. By injecting tiny, geometrically plausible errors into the reported position—well within the boundaries of normal GPS noise—the attacker forces the fusion pipeline to perform an incorrect coordinate transformation. This creates a systematic spatial shift, causing the ego vehicle to misalign the fused data, miss critical obstacles, or trigger unnecessary emergency braking. Because the data itself looks realistic, these attacks easily evade standard anomaly detectors.
3. Exploiting Uncertainty and Latency
The authors of Pretend Benign highlight that adversarial data is most dangerous when targeted at occluded or uncertain environmental regions. If the ego vehicle cannot independently verify a region using its own local sensors, it is forced to rely entirely on the malicious peer's contribution.
Furthermore, attacks do not only target spatial accuracy. In CP-FREEZER, Wang et al. showed that injecting adversarial perturbations can exploit the quadratic computational complexity of downstream post-processing algorithms (like non-maximum suppression). This triggers extreme computational overhead, increasing perception latency by over 90× and effectively "freezing" the vehicle's real-time decision-making capabilities.
The Defense Mechanics: Verifying Without Ground Truth
If an ego vehicle cannot physically see an occluded area, how can it determine if a peer's report is malicious? Current research focuses on two core strategies: consensus verification and latent feature consistency.
Consensus-Based Filtering
This strategy treats collaborative perception as a decentralized voting problem, operating on the principle that it is incredibly difficult for an attacker to generate falsified data that simultaneously achieves a malicious goal and remains perfectly consistent with the ground truth captured by benign agents.
For example, the ROBOSAC framework (Li et al.) applies a "hypothesize-and-verify" sampling strategy akin to RANSAC. It iteratively samples subsets of peer inputs to find a consensus group that aligns cleanly with the ego vehicle's independent local observations, discarding outliers. Similarly, CP-Guard (Hu et al.) utilizes a divide-and-conquer consensus algorithm to isolate malicious actors dynamically without needing prior knowledge of how many attackers are present in the network.
Feature-Level Consistency Verification
Waiting to verify final outputs (like 3D bounding boxes) is often too slow and misses stealthy adversarial noise. Modern defenses therefore shift verification directly into the latent feature space.
Systems like CP-Guard+ (Hu et al.) and Cerberus (Zhang et al.) implement feature-level consistency checks. By training networks with contrastive loss functions, the system learns to map legitimate, cross-vehicle observations of the same scene to tightly clustered representations in the latent space. If an attacker injects an adversarial perturbation or a fabricated object, the feature map deviates from this expected geometric and semantic harmony, allowing the ego vehicle to flag and isolate the malicious agent in real time.
Open Challenges
While adaptive weighting mechanisms (Chen et al.) can filter out random wireless channel noise, defending against intentional, coordinated manipulation remains an open challenge. Current cooperative frameworks frequently assume a homogeneous environment where all vehicles run identical machine learning models (Qu et al.), leaving them unprepared for the cross-model discrepancies of heterogeneous fleets.
Finally, detecting an attack is only half the battle. As noted by Zhang et al., the field still lacks robust, safe fallback and recovery protocols. Once an attack is identified, determining exactly how a vehicle should safely transition back to ego-only sensing without causing traffic disruptions or cascading signal failures (Li et al.) remains a critical area for future research.
Key Foundational Work
TruPercept: Trust Modelling for Autonomous Vehicle Cooperative Perception from Synthetic Data (2019) — contributed 2 concept elements to this direction.
Adversarial Attacks On Multi-Agent Communication (2021) — contributed 2 concept elements to this direction.
F-cooper: feature based cooperative perception for autonomous vehicle edge computing system using 3D point clouds (2019) — contributed 1 concept elements to this direction.