Visual Place Recognition (VPR) is a critical component for autonomous robot navigation, answering the fundamental question: "Where am I?" While outdoor VPR has achieved significant milestones, indoor environments present a unique challenge known as perceptual aliasing. Man-made structures often feature symmetrical designs, repetitive floor layouts, and identical decorative elements. To a standard VPR model, a hallway on the 2nd floor may appear indistinguishable from one on the 5th floor.
The TextInPlace Framework
The authors of TextInPlace propose that robots should emulate human navigation by utilizing textual clues—such as room numbers and floor signs—as unique semantic landmarks. To achieve this efficiently, the framework employs a dual-branch neural network architecture designed to balance accuracy with computational overhead.
Dual-Branch Architecture and Parameter Sharing
To maintain a lightweight profile, TextInPlace utilizes a Local Parameter Sharing Network. The system decomposes a pre-trained backbone (e.g., DINOv2 or ResNet) into two segments: early layers are frozen to preserve general features, while later layers are shared between the VPR and Scene Text Spotting (STS) branches. This design significantly reduces the total parameter count and inference time.
- VPR Branch (Coarse Retrieval): This branch employs a "Bag-of-Queries" (BoQ) approach with Multi-Head Attention to generate a global descriptor. It performs a K-nearest neighbor search to retrieve the top-K most similar images from a reference database.
- STS Branch (Fine-grained Verification): Simultaneously, this branch detects and recognizes text within the query image to provide high-level semantic context.
Discriminative Text Filtering
A major hurdle in text-based VPR is that not all text is informative. Generic signs like "Emergency Exit" or "Push" appear frequently and can actually increase aliasing. TextInPlace introduces two filtering strategies to isolate unique identifiers:
- Rule-based Filtering: A simple heuristic that prioritizes numerical digits, which are likely to represent room or floor numbers.
- LLM-based Filtering (TextInPlace-L): A more sophisticated approach using GPT-4o mini. The LLM acts as a zero-shot classifier, using "common sense" to distinguish between unique landmarks (e.g., "Mitte 501") and generic labels (e.g., "Fire Hydrant").
Following filtering, a re-ranking step calculates string similarity between the query text and the text in the top-K retrieved images, ensuring the most semantically relevant match is prioritized.
Experimental Results
The framework was evaluated on several datasets, including the newly introduced Maze-with-Text benchmark. This dataset contains over 9,800 images across five floors of a university building, specifically designed to test navigation in repetitive indoor spaces.
Key findings include:
- Significant Accuracy Gains: On the Maze-with-Text dataset, TextInPlace achieved a Recall@1 of 85.4%, a substantial improvement over the 67.5% baseline set by the appearance-only BoQ model.
- Robustness to Aliasing: In multi-floor scenarios where visual features were nearly identical, the integration of floor numbers allowed TextInPlace to achieve near-perfect disambiguation.
- The LLM Advantage: The LLM-based variant (TextInPlace-L) consistently outperformed the rule-based version, demonstrating the value of semantic reasoning in identifying reliable landmarks.
Limitations and Future Directions
While highly effective, the LLM-based re-ranking introduces a latency bottleneck, taking approximately 962ms compared to the 0.1ms of the rule-based approach. This makes the LLM variant less suitable for high-speed real-time robotics in its current form. Additionally, the system's performance is inherently tied to the presence of text; in featureless environments, it reverts to standard VPR performance. Future work will likely focus on optimizing on-device text processing to bridge the gap between semantic intelligence and real-time execution.
Summary updated