Abstract
Egocentric pose estimation is a fundamental capability for multi-robot collaborative perception in connected autonomy, such as connected autonomous vehicles. During multi-robot operations, a robot needs to know the relative pose between itself and its teammates with respect to its own coordinates. However, different robots usually observe completely different views that contains similar objects, which leads to wrong pose estimation. In addition, it is unrealistic to allow robots to share their raw observations to detect overlap due to the limited communication bandwidth constraint. In this paper, we introduce a novel method for Non-Overlap-Aware Egocentric Pose Estimation (NOPE), which performs egocentric pose estimation in a multi-robot team while identifying the non-overlap views and satisfying the communication bandwidth constraint. NOPE is built upon an unified hierarchical learning framework that integrates two levels of robot learning: (1) high-level deep graph matching for correspondence identification, which allows to identify if two views are overlapping or not, (2) low-level position-aware cross-attention graph learning for egocentric pose estimation. To evaluate NOPE, we conduct extensive experiments in both high-fidelity simulation and real-world scenarios. Experimental results have demonstrated that NOPE enables the novel capability for non-overlapping-aware egocentric pose estimation and achieves state-of-art performance compared with the existing methods.
Introduction
In collaborative autonomy—such as a fleet of self-driving cars at a busy intersection—robots must know exactly where their teammates are to merge their sensor data into a "shared" view of the world. This process, known as egocentric pose estimation, faces two primary hurdles. First, robots often observe non-overlapping views. Traditional algorithms can be misled by similar-looking objects (e.g., generic traffic signs) in different locations, leading to significant positioning errors. Second, sharing high-resolution raw data is infeasible due to the limited bandwidth of standard V2X (vehicle-to-everything) networks, which typically max out around 7.2 Mbps.
To address these challenges, we introduce NOPE (Non-Overlap-Aware Egocentric Pose Estimation), a hierarchical learning framework that enables robots to determine relative positions even when cameras see different parts of a scene. By representing observations as lightweight graphs, NOPE achieves a 96x reduction in communication bandwidth while improving pose accuracy by over 50% compared to current state-of-the-art methods.
Methodology: The NOPE Framework
The NOPE framework utilizes a two-tier hierarchical learning approach that treats each robot's view as a graph, where nodes represent detected objects and edges represent spatial relationships.
High-Level: Correspondence Identification (CoID)
Before calculating position, the system determines if the robots are observing the same area. It leverages Large Vision Models (LVMs) like DINO or CLIP to extract features and employs deep graph matching to identify physical correspondences. If no overlap is detected, the system avoids making a "hallucinated" guess, a critical safety feature for autonomous systems.
Low-Level: Position-Aware Graph Learning
When overlap is confirmed, a position-aware cross-attention network captures the holistic context of the scene. It uses the relative geometry of objects to calculate precise relative pose (distance and rotation) between robots. By sharing only object features and coordinates rather than raw pixels, the data packet size is reduced to just 27.0 KB, a 96.48% reduction compared to the 0.75 MB required by previous methods.
Experimental Evaluation
The model was evaluated in both virtual and physical environments:
- Simulation (CAD): Created using CARLA and SUMO, featuring five connected autonomous driving scenarios with over 30,000 instances.
- Real-World (MARS): Utilized multi-agent, multi-traversal data from a real fleet of autonomous vehicles equipped with LiDAR, cameras, and GPS.
NOPE significantly outperformed existing methods like CoViS-Net and SuperGlue. In simulations, it improved position error by 53.87% and rotation error by 78.64%. Furthermore, it successfully identified non-overlapping views with over 80% accuracy, preventing erroneous pose estimations.
Discussion and Future Work
The core innovation of NOPE is the realization that spatial arrangements of discrete objects matter more for pose estimation than specific textures. This abstraction enables massive bandwidth savings and robust generalization via LVM integration.
However, limitations remain. NOPE currently cannot estimate pose with zero overlap; future work may integrate Bayesian filters (like Kalman filters) to track positions during visual loss. Additionally, the researchers aim to expand the framework from robot pairs to larger teams to ensure "global consistency" and improve performance in highly dynamic or occluded urban environments.
For more details, visit the project page at https://hongh0.github.io/NOPE/.
Summary updated