Abstract
Using multiple robots for exploring and mapping environments can provide improved robustness and performance, but it can be difficult to implement. In particular, limited communication bandwidth is a considerable constraint when a robot needs to determine if it has visited a location that was previously explored by another robot, as it requires for robots to share descriptions of places they have visited. One way to compress this description is to use constellations, groups of 3D points that correspond to the estimate of a set of relative object positions. Constellations maintain the same pattern from different viewpoints and can be robust to illumination changes or dynamic elements. We present a method to extract from these constellations compact spatial and semantic descriptors of the objects in a scene. We use this representation in a 2-step decentralized loop closure verification: first, we distribute the compact semantic descriptors to determine which other robots might have seen scenes with similar objects; then we query matching robots with the full constellation to validate the match using geometric information. The proposed method requires less memory, is more interpretable than global image descriptors, and could be useful for other tasks and interactions with the environment. We validate our system's performance on a TUM RGB-D SLAM sequence and show its benefits in terms of bandwidth requirements.
Introduction
In Multi-Robot Simultaneous Localization and Mapping (SLAM), robots must recognize when they are visiting a place previously explored by a teammate—a process called collaborative place recognition. This is essential for "closing loops" and correcting the drift in their estimated trajectories. However, traditional methods often struggle with two major hurdles: high bandwidth consumption from sharing dense visual features and the risks associated with centralized server architectures. CAPRICORN addresses these by leveraging sparse, interpretable semantic data in a decentralized framework.
Methodology
The core innovation of CAPRICORN is representing scenes as 3D Semantic Constellations. Instead of raw pixels, the system uses 3D points representing the centers of detected objects (e.g., chairs, bottles, laptops).
As illustrated in the figure above, the system operates in a two-step decentralized pipeline:
- Coarse Semantic Matching: Each robot generates a "Semantic Descriptor"—a histogram of object types. To maintain decentralization, specific robots are assigned "responsibility" for specific object classes. A querying robot sends its partial descriptors only to the robots responsible for the objects it currently observes.
- Fine Geometric Verification: The querying robot sends its full 3D constellation (relative object positions) to the best candidates identified in the first step. The receiving robot then verifies the match using a geometric surroundings descriptor to ensure spatial consistency.
Evaluation and Results
The system was validated using the TUM RGB-D SLAM dataset (freiburg3_long_office_household sequence). The centralized version of CAPRICORN achieved an Area Under the Curve (AUC) of 0.9282, while the decentralized version reached 0.8482, demonstrating that robust place recognition is possible even with the constraints of a distributed network.
Communication efficiency is a primary strength of this approach. As shown in the figure above, the query size for a place recognition check is approximately 490 Bytes. This is comparable to highly optimized visual descriptors like NetVLAD but offers superior scalability in multi-robot networks because the initial semantic "handshake" is even smaller, preventing network congestion as the swarm grows.
Discussion and Future Work
CAPRICORN demonstrates that semantic sparsity is a powerful tool for multi-robot coordination. By distributing responsibility for object classes, the system avoids bottlenecks. Furthermore, the use of constellations provides natural robustness to viewpoint changes and offers high interpretability for human operators.
However, the system's performance is currently dependent on the accuracy of the underlying object detector (YOLOv3). Future research will focus on dynamic load balancing to handle uneven object distributions in different environments and improving the precision of relative pose estimation to match the performance of traditional visual features for fine-grained coordinate transforms.
Summary updated