Connected vehicles can see around occlusions by sharing what their sensors perceive. Wireless links are far too narrow for raw sensor data, so the systems that work in practice share a small, selected subset of intermediate neural features. These systems choose what to send using a detector trained on a fixed list of classes such as cars and pedestrians. That choice has a blind spot. A tire, a fallen load or a spill on the road is not on the list, so the detector gives it near-zero confidence and the sender drops exactly the evidence the receiver most needed. We propose to choose what to transmit with a second signal: how badly a model trained only on normal driving scenes fails to reconstruct each region of the shared bird's-eye-view map. Here "bird's-eye-view (BEV)" means a top-down grid of features centered on the vehicle.
The two halves of the idea fit together because of a known weakness of reconstruction-based anomaly detection. Small or distant hazards leave little reconstruction error and are often missed. A collaborator that happens to be close to the hazard sees it large and clearly, so its error is high. Gating transmission on reconstruction error therefore sends the right evidence from the agent best placed to see it. Because the selected token sets are small, the receiver can align them with the other agent's tokens using optimal transport, and do so exactly, without the approximations that dense transport problems need. Optimal transport here means finding the lowest-cost matching between two sets of points.
Core Claim
At a fixed transmission budget, adding a reconstruction-residual trigger to confidence-based feature selection will substantially raise the receiving vehicle's recall of out-of-class road hazards. It will do so without lowering in-class detection accuracy. A partial optimal-transport fusion step, which is allowed to leave tokens unmatched, will further improve hazard localization under realistic pose error and transmission delay. We treat the first part as the main claim. The second part is a secondary claim that we will drop if its ablation fails.
Where the Gap Is
The closest work on choosing what to send motivates this proposal by showing the gap. Where2comm sends only features from regions where a supervised detection decoder is confident, and broadcasts "request maps" that mark the receiver's blind spots. It reports up to 100,000× less communication on OPV2V. Its confidence comes from a closed-set box decoder, however, so unknown objects score low and are discarded. COOPERTRIM selects features that deviate from a temporal memory of earlier fused states. It reaches 1.46% of the original bandwidth when combined with 32× compression. Its notion of novelty is change over time, not departure from what normal roads look like. A static hazard that has sat in view for several frames is no longer novel to it. CoBEVT makes multi-agent BEV segmentation efficient with sparse axial attention, but it segments a fixed taxonomy. Background pruning explicitly removes uninformative background tokens, and an unlabeled hazard can be mistaken for background. None of these methods models normality.
The reconstruction side supplies the method and also limits it. JSR-Net reconstructs only the road surface through a narrow bottleneck and couples the reconstruction error with segmentation logits, reaching 94.4% AP on Road Anomaly. It is single-vehicle and camera-only. We borrow its bottleneck-plus-coupling design and move it from image pixels to BEV features. DRÆM prevents the "identical shortcut", where the reconstruction simply copies the anomaly. It does this by training a discriminative head on input–reconstruction pairs with synthetic defects pasted in, and we reuse that recipe. Fishyscapes and JSR-Net both report that small, distant hazards remain hard. That is a negative result for single-agent reconstruction, and we turn it into the motivation for collaboration.
On the transport side, mini-batch partial optimal transport shows that moving only a fraction $s$ of the mass avoids forced, wrong pairings. It fixes $s$ by grid search and is evaluated only on domain adaptation. Low-rank Sinkhorn and positive-feature Sinkhorn make dense transport roughly linear-time. They are the right tools if one tried to match full BEV grids of tens of thousands of cells. We include them as a comparison point, not as the method. Two works pair optimal transport with anomaly detection, MADPOT and OT-calibrated tabular detection. Both are single-source and have no communication constraint.
Research Program
Each agent runs a shared BEV encoder, for example the LiDAR or camera BEV backbones used on OPV2V. Attached to the frozen encoder is a normalcy module. This module is a bottlenecked autoencoder over BEV feature patches, trained only on scenes with no out-of-taxonomy objects. Alongside it sits a DRÆM-style discriminative head. The head sees each feature, its reconstruction and the closed-set segmentation logits, and it is trained with synthetic anomalies, meaning randomly shaped feature-space blobs and pasted point-cloud props. Its output is a per-cell anomaly score $a_i$.
The sender ranks cells by a combined priority
$$ p_i = \max\left(c_i,\ \lambda\, a_i \right)\cdot r_i , $$
Here $c_i$ is the Where2comm-style closed-set confidence of cell $i$, $a_i$ is the anomaly score and $\lambda$ balances the two. The term $r_i$ is the receiver's request value for that cell, its inverse confidence as in Where2comm. The sender then transmits the top-$k$ cells that fit the byte budget. In every frame it also sends a 16× downsampled map of $a_i$. That map costs a few hundred bytes, and it lets the receiver ask for more detail where a collaborator's anomaly score is moderate.
At the receiver, the incoming sparse tokens $\{y_j\}$ are aligned with the ego vehicle's own high-score tokens $\{x_i\}$. The alignment solves an entropic partial transport problem over a cost that mixes feature distance with position under the reported pose. The transported mass fraction $s$ is set per frame to the estimated overlap of the two agents' anomaly maps, so that tokens seen by only one agent stay unmatched instead of being forced onto a wrong partner. Matched pairs are fused, and unmatched collaborator tokens are inserted at their reported positions. With $k$ in the low hundreds, Sinkhorn on a $k \times k$ matrix costs well under a millisecond. Running the same alignment on the dense grid would need low-rank or positive-feature solvers. The sparsity that saves bandwidth therefore also makes exact optimal transport affordable.
Benchmarks and Data
No collaborative anomaly benchmark appears in the literature we have, so we will build one. OPV2V is generated in CARLA with OpenCDA. Re-running its scenario scripts with out-of-taxonomy props on drivable lanes gives perfect anomaly masks from every agent's viewpoint. The props are debris, boxes, fallen cargo and animals, placed in occluded, distant and near positions. In-class evaluation uses the standard OPV2V and V2V4Real splits, so we can check that the trigger does not hurt ordinary detection. V2V4Real has no anomaly labels. As a real-world proxy, we will hold out the rarest annotated class or any objects that fall outside the box taxonomy, and report this proxy clearly as weaker evidence. Because our module transfers the JSR-Net design to BEV features, we will also run its camera branch single-agent on Road Anomaly and Fishyscapes Lost & Found. That check confirms the normalcy module is a competent detector before we judge its value as a transmission trigger.
The metrics are BEV anomaly average precision and the false-positive rate at 95% recall, both measured at the receiver. We split results by hazard visibility, separating hazards visible to the ego vehicle from those occluded or distant to it. We also report in-class [email protected] and IoU, plotted against bytes per frame, and end-to-end latency.
Baselines
The naive reference point is no collaboration: the ego vehicle runs the same normalcy module alone. This isolates how much the collaborator adds for occluded and distant hazards. At the other extreme sits dense intermediate fusion of full BEV maps, the OPV2V pipeline, with the anomaly head applied after fusion. It is an upper bound that ignores bandwidth.
The strongest existing competitor on the bandwidth axis is Where2comm. It is the most aggressive selection method in our sources, and its request maps already target blind spots. The best collaborative detector reported on V2V4Real is CoBEVT, at 66.5% [email protected] according to the V2V4Real paper. We use CoBEVT as the fusion backbone wherever camera BEV is involved. COOPERTRIM is the strongest competitor on the novelty axis, because temporal non-conformity is the closest existing signal to "something unusual is here."
All selection methods will share the same encoder, the same fusion network and the same downstream anomaly head. Only the rule that picks which cells to send will differ. They will be compared at matched byte budgets of roughly 1, 4 and 16 kilobytes per frame, with an equal hyperparameter search budget for each.
We found no reported result on Lacuna for anomaly AP under bandwidth limits for any collaborative method. All anomaly numbers on our split will therefore be new, and the in-class numbers must stay within one AP point of Where2comm at equal bytes. Beating the baselines means a statistically clear gain in anomaly AP on hazards occluded from or distant to the ego vehicle, at every budget of 4 kilobytes or less, with in-class accuracy preserved.
Three ablations isolate the key ingredients. The first is random selection at the same budget. The second replaces the anomaly score $a_i$ with the decoder's own predictive entropy. The third keeps residual-triggered selection but fuses with plain warp-plus-attention instead of partial transport, which isolates whether the transport step adds anything.
Why not just…?
Why not just use the detector's uncertainty, such as softmax entropy, as the trigger? Fishyscapes evaluates entropy and Bayesian dropout as anomaly signals and finds them weaker than dedicated outlier modeling. JSR-Net's large margin over prior methods on Road Anomaly, 94.4% against about 76%, comes from adding reconstruction error to the logits. That evidence comes from single-agent images, though. The entropy-trigger ablation decides the question in our setting. If entropy matches the anomaly score at equal bytes, the normalcy module is not worth its compute, and we will say so.
Why not just rely on COOPERTRIM's temporal novelty? A hazard is temporally novel only for the first frames in which it appears. After that it enters the memory bank and stops being sent, even though a newly arriving collaborator still needs it. We will build a dedicated subset of static, long-dwelling hazards to test this. If COOPERTRIM performs comparably there, then temporal and normalcy novelty are redundant, and the proposal narrows to combining the two signals.
Why not just have each agent detect anomalies locally and send compact score maps, a late-fusion approach? This is the most serious threat, because a low-resolution score map is extremely cheap. Its weakness, by hypothesis, is that a distant hazard falls below threshold at each agent separately. Fusing features before thresholding lets evidence from both agents add up. The score-map-only variant is already part of our system, as the always-sent downsampled map. Running it alone at equal bytes measures exactly how much the transmitted features add. If it matches the full method, we will change the claim to "share normalcy residual maps, not features," which would still be a useful and simpler result.
Why optimal transport rather than warping by reported pose and fusing with attention? If poses are accurate, warping is enough. The V2V4Real paper reports that 100 ms delays notably degrade fusion, and misalignment is the regime where a matching step that can leave tokens unpaired should help. We will inject controlled pose noise and delay. If partial transport does not beat warp-plus-attention under those conditions, we will drop it, and the main claim stands on selection alone.
Decisive Checks
The proposal succeeds or fails on one plot: anomaly AP for hazards occluded from the ego vehicle, against bytes per frame, comparing Where2comm, COOPERTRIM, late-fusion score maps and our trigger. Our curve must sit clearly above the others at low budgets while the in-class curve matches Where2comm. A second check sorts hazards by their distance to the ego vehicle and to the nearest collaborator. The "collaborator sees it larger" hypothesis predicts that our gain grows with the gap between those distances. If it does not, the gain must come from somewhere else and needs another explanation. A third check measures latency on a single GPU. We will report whether the normalcy module and the $k\times k$ transport keep the frame-rate cost below COOPERTRIM's reported drop of about 2 FPS. We will also compare against low-rank Sinkhorn on the dense grid to make the scalability argument concrete.
References
- Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps
- COOPERTRIM: Adaptive Data Selection for Uncertainty-Aware Cooperative Perception
- CoBEVT: Cooperative Bird’s Eye View Semantic Segmentation with Sparse Transformers
- Background Fades, Foreground Leads: Curriculum-Guided Background Pruning for Efficient Foreground-Centric Collaborative Perception
- Road Anomaly Detection by Partial Image Reconstruction with Segmentation Coupling
- DRÆM - A discriminatively trained reconstruction embedding for surface anomaly detection
- The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation
- Improving Mini-batch Optimal Transport via Partial Transportation
- Low-Rank Sinkhorn Factorization
- Linear Time Sinkhorn Divergences using Positive Features
- MADPOT: Medical Anomaly Detection with CLIP Adaptation and Partial Optimal Transport
- Calibrating Tabular Anomaly Detection via Optimal Transport
- OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication
- V2V4Real: A Real-World Large-Scale Dataset for Vehicle-to-Vehicle Cooperative Perception