CoMET: Tracking the Shared Pose Error of Roadside Messages for Asynchronous Vehicle-Infrastructure Cooperative 3D Detection

Shuwei Sun, Jian Huang, Hui CaoarXiv 2026 · arXiv

Cooperative perception—where vehicles and roadside infrastructure (like LiDAR-equipped poles) share data—promises to eliminate blind spots, such as a car hidden behind a large truck. However, real-world deployment faces two major hurdles: latency (messages arrive late) and pose error (misalignment between the vehicle and the roadside sensor).

In their paper, CoMET, researchers from Xi’an Jiaotong University demonstrate that the geometric misalignment (pose error) is actually a much bigger bottleneck than communication delay. They propose a elegant, tracking-based solution that significantly outperforms existing complex fusion models.

TL;DR

CoMET (Common-Mode Error Tracking) treats the coordinate misalignment between vehicle and infrastructure as a persistent, drifting state rather than a per-frame registration problem. By using a causal Kalman filter to track this shared translation offset, CoMET aligns roadside data even when common objects are temporarily invisible, boosting 3D detection accuracy by over 3.4 AP points.

What Problem Does This Paper Address?

When a roadside unit (RSU) sends a message to a vehicle, every object in that message is typically shifted by the same "common-mode" translation error due to imperfect calibration or localization.

Existing methods usually try to "re-register" every frame from scratch. This fails in two scenarios:

  1. Sparsity: In many frames (38% in the studied dataset), there are no "anchors" (objects seen by both the car and the RSU) to calculate the shift.
  2. Noise: Per-frame estimates are jittery and unreliable.

The authors also highlight a "strict arrival protocol": in a real system, you can't use future frames or ground-truth offsets to fix your current view. You only have what has already arrived.

Methods

CoMET treats the translation error as a latent state in a message stream. The pipeline works as follows:

  1. Anchor Matching: It uses the Hungarian algorithm to match detections from the vehicle (ego) and the RSU. These matched pairs are "anchors" that reveal the current translation residual.
  2. Causal Kalman Filtering: Instead of a one-off calculation, CoMET uses a scalar Kalman filter to track the offset per sender.
  3. If anchors are present, it updates the estimate.
  4. If no anchors are visible (e.g., the car is in a quiet area), the filter carries the last known estimate forward while increasing the uncertainty (variance).
  5. Error Transfer: The tracked offset is applied to all objects in the RSU message—including the "hidden" cars that the ego vehicle cannot see.
  6. Frozen Readout Head: After merging the corrected RSU boxes with the ego detections, a tiny, 1.3K-parameter MLP (trained once and then frozen) re-ranks the candidates to produce the final output.
Figure 1: CoMET at a glance. (a) A roadside message is admissible only if it was captured at least d ms before the decision time t. Placed with online poses, all remote boxes (red, dashed) are displaced from the ground truth (gray) by the same offset δt (exaggerated here). Objects seen by both agents yield anchor residuals zk between ego boxes (blue) and remote boxes that measure δt; the tracked estimate moves every remote box, including the roadside-only objects hidden behind the truck (green). (b) A real validation stream (sequence 0071, d = 0). Anchors per message come from Hungarian matching of all ego detections with a mutual-nearest-neighbour gate. The estimate from the current message alone (orange) is noisy and absent when there are no anchors (gray bands); the CoMET estimate (blue) is stable, is kept through anchor-free messages and its uncertainty grows until anchors return. (c) Confident ego detections form a protected prefix E+ t ; corrected remote boxes and weak ego boxes Ct are ranked by a 1.3K-parameter readout head on 39 receiver-side features without absolute position, trained once and frozen. Only boxes, scores and capture times are transmitted.

Datasets

The study primarily uses V2X-Seq, a large-scale sequential dataset for vehicle-infrastructure cooperation.

A key contribution of the paper is a rigorous audit of the V2X-Seq benchmark. The authors discovered that the "reference offsets" provided by the dataset are often flawed: 28% are zero-value placeholders, and 45% of the rest are "saturated" (clipped at a search boundary). This means standard evaluations might actually penalize methods that find the true physical offset.

Key Insights and Ideas

  • Geometry > Latency: Improving pose alignment yields a ~7.4 point gain in AP, while reducing latency from 400ms to 0ms only gains ~2.3 points. Solving the "where" is more important than the "when."
  • Tracking vs. Registering: By treating the error as persistent, CoMET maintains alignment through "dark" periods where no common objects are visible. This is the "aha moment": the error doesn't vanish just because you can't see an anchor right now.
  • Simplicity Wins: The pose estimator has zero learned parameters. It relies on classical filtering theory, making it robust and computationally cheap.
Figure 3: One validation stream (sequence 0007) at two latency budgets. Orange: offset estimated from the anchors of the current message only (shown only when the message has anchors). Blue: the CoMET estimate with its ±2σ band; gray bands mark messages without anchors. Bottom: number of anchors per message. The per-message estimate is noisy and disappears when anchors are absent (after about 9 s); the tracked estimate is stable, follows the change in y around 8 s, and keeps the last estimate with growing uncertainty when anchors vanish. At 400 ms the per-message estimate is noticeably noisier.

Key Results

  • Superior Accuracy: CoMET achieves 66.63 [email protected] on the validation set, which is 3.45 points higher than uncorrected late fusion and 3.67 points higher than the best intermediate-fusion baseline.
  • Consistency: It improves performance across five different detector architectures (including V2X-ViT and Where2comm).
  • Latency Resilience: Even with a 400ms delay, CoMET outperforms other methods operating at 0ms delay.
Figure 2: [email protected] against the latency budget on V2X-Seq validation. Intermediate fusion and late fusion with online poses stay close to the ego detector. CoMET recovers over half of the gap to the upper reference that places the boxes with the dataset’s annotation-assisted offset (dashed gray, not deployable). Values are means over three readout heads or three detector seeds; numbers in Table 2.

Limitations and Future Work

  • Planar Translation Only: The current model assumes the error is a 2D translation. While this covers the majority of the error in V2X-Seq, it does not explicitly account for rotation or elevation errors.
  • Gate Sensitivity: The system uses a 2.5m gate for matching anchors. If the initial error is larger than this gate (a "cold start" problem), the system cannot acquire the first anchor.
  • Future Directions: The authors suggest extending the filter to include explicit rotation terms and weighting anchors based on their motion or distance.

Summary updated

Abstract

A roadside sensor can detect vehicles hidden from the ego vehicle, but a pose error shifts every detection in the same message. We study asynchronous vehicle-infrastructure cooperative 3D detection on V2X-Seq using only information available at decision time, without ground truth, future frames, or per-pair calibration offsets at inference. Objects seen by both sensors reveal the shared translation, but even at zero added delay, 38% of validation messages have no matched pair. We propose CoMET (Common-Mode Error Tracking), which estimates this translation over time. At each message arrival, CoMET matches vehicle and roadside detections, estimates the offset from mutually nearest pairs, and updates a Kalman filter for that sender. When no pair matches, the filter carries its estimate forward. The correction applies to all roadside boxes, including those unseen by the ego vehicle. The estimator has no learned parameters; a separate frozen head ranks the merged detections. Across four delays from 0 to 400 ms on 21 validation sequences, CoMET averages 66.63 bird's-eye-view AP at IoU 0.7, compared with 63.18 for the same late fusion using uncorrected online poses and 62.96 for the best intermediate-fusion baseline. Across five detectors from three architectures, CoMET gains 2.2 to 3.5 AP points over online poses and 0.3 to 0.6 over estimating the offset from the current message alone. We find that the benchmark uses a zero offset as a placeholder in 28% of validation messages and that its offset search reaches the boundary in 45% of the rest. These offsets affect the labels for objects seen only from the roadside. Under two alternative label references, CoMET improves on uncorrected online poses in every evaluated population. Retraining intermediate fusion with CoMET poses and corrected labels improves detection on all three architectures tested, with statistically significant gains on two.
BibTeX
@misc{sun2026comet,
  author = {Shuwei Sun and Jian Huang and Hui Cao},
  title = {{CoMET: Tracking the Shared Pose Error of Roadside Messages for Asynchronous Vehicle-Infrastructure Cooperative 3D Detection}},
  year = {2026},
  eprint = {2611.09121},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09121},
}
Figure