Cooperative perception—where vehicles and roadside infrastructure (like LiDAR-equipped poles) share data—promises to eliminate blind spots, such as a car hidden behind a large truck. However, real-world deployment faces two major hurdles: latency (messages arrive late) and pose error (misalignment between the vehicle and the roadside sensor).
In their paper, CoMET, researchers from Xi’an Jiaotong University demonstrate that the geometric misalignment (pose error) is actually a much bigger bottleneck than communication delay. They propose a elegant, tracking-based solution that significantly outperforms existing complex fusion models.
TL;DR
CoMET (Common-Mode Error Tracking) treats the coordinate misalignment between vehicle and infrastructure as a persistent, drifting state rather than a per-frame registration problem. By using a causal Kalman filter to track this shared translation offset, CoMET aligns roadside data even when common objects are temporarily invisible, boosting 3D detection accuracy by over 3.4 AP points.
What Problem Does This Paper Address?
When a roadside unit (RSU) sends a message to a vehicle, every object in that message is typically shifted by the same "common-mode" translation error due to imperfect calibration or localization.
Existing methods usually try to "re-register" every frame from scratch. This fails in two scenarios:
- Sparsity: In many frames (38% in the studied dataset), there are no "anchors" (objects seen by both the car and the RSU) to calculate the shift.
- Noise: Per-frame estimates are jittery and unreliable.
The authors also highlight a "strict arrival protocol": in a real system, you can't use future frames or ground-truth offsets to fix your current view. You only have what has already arrived.
Methods
CoMET treats the translation error as a latent state in a message stream. The pipeline works as follows:
- Anchor Matching: It uses the Hungarian algorithm to match detections from the vehicle (ego) and the RSU. These matched pairs are "anchors" that reveal the current translation residual.
- Causal Kalman Filtering: Instead of a one-off calculation, CoMET uses a scalar Kalman filter to track the offset per sender.
- If anchors are present, it updates the estimate.
- If no anchors are visible (e.g., the car is in a quiet area), the filter carries the last known estimate forward while increasing the uncertainty (variance).
- Error Transfer: The tracked offset is applied to all objects in the RSU message—including the "hidden" cars that the ego vehicle cannot see.
- Frozen Readout Head: After merging the corrected RSU boxes with the ego detections, a tiny, 1.3K-parameter MLP (trained once and then frozen) re-ranks the candidates to produce the final output.
Datasets
The study primarily uses V2X-Seq, a large-scale sequential dataset for vehicle-infrastructure cooperation.
A key contribution of the paper is a rigorous audit of the V2X-Seq benchmark. The authors discovered that the "reference offsets" provided by the dataset are often flawed: 28% are zero-value placeholders, and 45% of the rest are "saturated" (clipped at a search boundary). This means standard evaluations might actually penalize methods that find the true physical offset.
Key Insights and Ideas
- Geometry > Latency: Improving pose alignment yields a ~7.4 point gain in AP, while reducing latency from 400ms to 0ms only gains ~2.3 points. Solving the "where" is more important than the "when."
- Tracking vs. Registering: By treating the error as persistent, CoMET maintains alignment through "dark" periods where no common objects are visible. This is the "aha moment": the error doesn't vanish just because you can't see an anchor right now.
- Simplicity Wins: The pose estimator has zero learned parameters. It relies on classical filtering theory, making it robust and computationally cheap.
Key Results
- Superior Accuracy: CoMET achieves 66.63 [email protected] on the validation set, which is 3.45 points higher than uncorrected late fusion and 3.67 points higher than the best intermediate-fusion baseline.
- Consistency: It improves performance across five different detector architectures (including V2X-ViT and Where2comm).
- Latency Resilience: Even with a 400ms delay, CoMET outperforms other methods operating at 0ms delay.
Limitations and Future Work
- Planar Translation Only: The current model assumes the error is a 2D translation. While this covers the majority of the error in V2X-Seq, it does not explicitly account for rotation or elevation errors.
- Gate Sensitivity: The system uses a 2.5m gate for matching anchors. If the initial error is larger than this gate (a "cold start" problem), the system cannot acquire the first anchor.
- Future Directions: The authors suggest extending the filter to include explicit rotation terms and weighting anchors based on their motion or distance.
Summary updated