Deep Masked Graph Matching for Correspondence Identification in Collaborative Perception

Peng Gao, Qingzhao Zhu, Hongsheng Lu, Chuang Gan, Hao Zhang

Introduction

Collaborative perception allows connected autonomous vehicles (CAVs) and robotic systems to share sensor data, effectively extending their sensing range and overcoming individual occlusions. A fundamental prerequisite for this collaboration is Correspondence Identification (CoID)—the process of determining whether objects detected by different agents represent the same physical entity. This paper presents Deep Masked Graph Matching (DMGM), a robust framework designed to solve CoID in complex, noisy environments by leveraging both visual features and spatial geometry.

The Challenges of Correspondence Identification

Identifying correspondences between disparate sensor views is complicated by three primary factors:

  1. Perceptual Aliasing: In urban environments, many objects (e.g., white sedans, traffic signs) appear identical, making visual-only matching unreliable.
  2. Non-covisibility: Agents often have partially overlapping fields of view. An object visible to one vehicle may be occluded or out of range for another, leading to "one-sided" detections that should not be matched.
  3. Sensor Noise: Inaccuracies in GPS and depth sensors introduce spatial discrepancies, preventing simple coordinate-based matching.

Deep Masked Graph Matching (DMGM)

DMGM addresses these challenges by formulating CoID as a graph matching problem, prioritizing the relative layout of objects over raw appearance.

Graph Representation and Embedding

Each agent constructs a graph where nodes represent detected objects. Nodes are initialized with Histogram of Oriented Gradients (HOG) features to capture visual appearance, while edges are established using Delaunay triangulation to represent the spatial relationships between objects.

The system employs an Attentional Graph Neural Network (GNN) to generate embeddings. By using a self-attention mechanism, the model encodes not just the object's appearance but also its structural context—how it relates to its neighbors. This ensures that even if two cars look identical, their unique positions relative to nearby landmarks allow for successful differentiation.

GPS Masking and Consensus

To incorporate geometric priors, DMGM utilizes a "GPS mask." Unlike methods that treat GPS as a standard feature, DMGM uses a reciprocal distance function to reward matches that are physically close in global coordinates. This acts as a strong constraint that persists even when visual data is degraded by weather or lighting. Furthermore, a graph consensus mechanism prunes matches that violate the global structural consistency between the two graphs.

Handling Non-covisibility

A key innovation in DMGM is the Non-covisible Remover. To handle objects that appear in only one agent's view, the system applies SoftMax variance thresholding. If the network's matching confidence for a node exhibits low variance across potential candidates (indicating high uncertainty), the system identifies the object as non-covisible and removes it from the correspondence matrix to prevent false positives.

Experimental Evaluation

The authors evaluated DMGM using a high-fidelity simulation environment combining CARLA (for visual and sensor simulation) and SUMO (for realistic traffic flow).

Dataset

The researchers generated a large-scale dataset comprising 69,469 data instances across five diverse intersection scenarios. This dataset is unique in providing ground-truth correspondences derived from instance-level semantic segmentation, offering a higher level of precision than existing collaborative perception datasets like OPV2V or V2X-Sim.

Performance and Robustness

DMGM significantly outperformed existing state-of-the-art methods, including GCN-GM, DGMC, and BDGM:

  • Superior Accuracy: DMGM achieved an F1 score of 0.8063 (Precision: 0.7859, Recall: 0.8278), compared to the next best method (BDGM), which achieved an F1 of 0.6437.
  • Noise Tolerance: The model demonstrated remarkable robustness; while GPS noise (increasing from 1.2m to 3m) impacted performance, it only caused a minor 2.5% decrease in the overall F1 score.
  • Real-time Efficiency: The system operates at approximately 20Hz on standard hardware, meeting the real-time requirements of autonomous driving applications.

Conclusion and Future Work

DMGM provides a robust solution for CoID by prioritizing structural layout over visual appearance and introducing an elegant mechanism for handling non-covisible objects. While the results in high-fidelity simulation are promising, future research should focus on real-world deployment to address potential network latency and explore temporal tracking to further stabilize correspondences in dynamic scenes.

Summary updated

BibTeX
@inproceedings{gao2023deep,
  author = {Peng Gao and Qingzhao Zhu and Hongsheng Lu and Chuang Gan and Hao Zhang},
  title = {{Deep Masked Graph Matching for Correspondence Identification in Collaborative Perception}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2023},
  doi = {10.1109/icra48891.2023.10161231},
}
Figure