TL;DR
Kolibri is a lightweight localization method that splits the task of finding a camera's pose into two steps: a tiny neural network uses color to find the approximate "place," and standard geometric refinement handles the precision. By leveraging the insight that many nearby initial guesses converge to the same correct pose, Kolibri achieves state-of-the-art accuracy while being 18x to 296x faster than existing methods, enabling real-time localization across entire buildings without pre-defined room labels.
What Problem Does This Paper Address?
Localization—the ability of a robot or agent to determine its exact position and orientation within a pre-mapped environment—is a fundamental challenge in robotics. While we have excellent tools for this, they often face a "speed vs. scale" trade-off.
Standard point cloud registration methods are highly accurate but computationally expensive, often taking seconds or even minutes per frame. To speed things up, many systems use a two-step "retrieve-then-align" workflow: first, an image retrieval system guesses which room the robot is in, and then a registration algorithm aligns the frame within that specific room. However, if the retrieval step picks the wrong room (which happens often in buildings with similar-looking hallways or offices), the localization fails entirely.
The authors of Kolibri set out to create a system fast enough to search an entire building's point cloud at once, removing the need for error-prone room labels.
Methods
The core philosophy of Kolibri is: "Color finds the place, refinement sets the pose."
- The Kolibri Network: Instead of a massive model, the authors use a tiny graph neural network (GNN) with only 55,712 parameters. It processes the environment as a graph of 10cm cubic "cells." Each cell stores information about its color, local shape, and its relationship to 16 neighboring cells.
- Candidate Proposing: The network matches cells from the current camera frame to cells in the global point cloud. It uses these matches to propose a few (up to 6) candidate poses.
- Geometric Refinement: Kolibri then hands these candidates over to a standard, non-learned algorithm: point-to-plane Iterative Closest Point (ICP).
The key technical insight is the "refinement basin." The authors observed that if an initial pose guess is within a certain distance (about 7.7cm) of the truth, ICP will almost always converge to the correct final pose. This means the learned neural network doesn't need to be perfectly accurate; it just needs to land "inside the basin."
Datasets
The researchers evaluated Kolibri on several standard benchmarks and one new real-world scenario:
- ScanNet: A large-scale dataset of indoor scenes. They used specific subsets (ScanNet-10, ScanNet-96) to test generalization to unseen rooms.
- 7-Scenes & 12-Scenes: Classic benchmarks for indoor relocalization.
- Hospital Emergency Department: A new, challenging test case involving a real-world scan of a large hospital wing (49m x 42m) captured with a handheld laser scanner.
Key Insights and Ideas
- Efficiency through Simplicity: By offloading the "heavy lifting" of precision to ICP, the neural network can remain incredibly small. This makes it feasible to run on low-power hardware or process massive environments quickly.
- Color as a Primary Cue: Much like humans use the "gist" of a scene (colors and rough shapes) to recognize a room, Kolibri uses color to narrow down the search space rapidly.
- Searching the Whole Building: Because Kolibri is so fast (82ms per frame), it can search a point cloud of 100 rooms simultaneously. This is much more robust than "retrieve-then-align" methods, which lose significant accuracy if the initial room retrieval is wrong.
Key Results
- Speed: On an RTX 2080 Ti, Kolibri processes frames in 82ms, making it roughly 18x faster than BUFFER-X and 296x faster than GeDi.
- Accuracy: Despite its small size, it matches the accuracy of much larger SOTA models, achieving 81.4% recall (within 5cm and 5°) on unseen ScanNet rooms.
- Scale: In the hospital emergency department test, Kolibri localized 86.5% of frames in 384ms each. For comparison, the GeDi method required 91 minutes per frame to achieve similar results on the same hardware.
Limitations and Future Work
The primary limitation of Kolibri is its dependence on color consistency.
- Grayscale/Low Light: If the camera provides grayscale images or if the lighting is drastically different from the original scan, performance drops significantly. In these cases, purely geometric methods like RGB GeoTransformer still hold the lead.
- Cross-Camera Calibration: The method is sensitive to differences in how different cameras respond to color.
Future work may involve creating more robust color-invariant features or developing a hierarchical approach to scale the system to even larger environments, like multi-story city blocks.
Summary updated