Finding a bed in a new house is usually easy for humans because we understand the "semantic recipe" of a home: start in the living room, go through a hallway, and you’ll likely find a bedroom. For embodied AI agents, however, this kind of cross-scene experience reuse is surprisingly difficult.
A new paper, HTCE, introduces a clever topological trick to help robots "canonicalize" their memories, allowing them to apply lessons learned in one apartment to a completely different one.
TL;DR
HTCE (Homeomorphism-inspired Topological Cross-Scene Experience) uses principles from topology to strip away scene-specific noise from navigation graphs, leaving behind a "canonical" structural signature. By matching these signatures, agents can retrieve successful historical paths to guide a frozen vision-language model (VLM) in unfamiliar environments, achieving state-of-the-art performance on major navigation benchmarks.
What Problem Does This Paper Address?
In Object-Goal Navigation (ObjectNav), an agent must find a specific object (e.g., a "bed") in a new environment. As agents explore, they build "experience graphs" where nodes represent regions and edges represent transitions.
The challenge is that two identical semantic paths can look completely different to a computer:
- Naming Variations: One scene might label a passage a "hallway," while another calls it an "entryway" or "corridor."
- Granularity: One robot might record five intermediate points while walking down a long hall, while another records only two.
Because of these representational differences, an agent often fails to realize that its current situation matches a successful strategy it used in a previous house.
Methods
The authors turn to graph homeomorphism. In topology, two graphs are homeomorphic if one can be transformed into the other by adding or removing "degree-two" vertices (points that sit in the middle of a path without branching).
HTCE applies this via a three-step pipeline:
- Feature Canonicalization: It partitions region labels into anchors (semantically important rooms like "kitchen" or "bedroom") and transitions (generic connectors like "hallway"). All transition labels are normalized to a single role.
- Topological Reduction: It suppresses intermediate transition nodes that don't serve as junctions. If you are moving from a living room to a bedroom through a hallway, HTCE doesn't care if that hallway has one node or ten—it reduces them to a single "transition" edge.
- Subdivision-Aware Matching: When the agent is in a new scene, it generates a "query" based on its current path. It then searches its persistent memory for historical "successful approaches" that have the same reduced structural signature.
This retrieved experience is then fed as a semantic cue to a frozen Vision-Language Model (VLM), specifically Qwen3-VL-Plus, which handles the final action selection.
Datasets
The researchers evaluated HTCE on three widely used embodied AI benchmarks:
- HM3D v0.1 & v0.2 (Habitat-Matterport 3D): Large-scale 3D residential environments.
- MP3D (Matterport3D): Diverse indoor environments with rich semantic annotations.
Key Insights and Ideas
- Lossless Compression: Perhaps the most striking finding is that this reduction is "lossless." By merging structurally identical but descriptively different routes, HTCE compresses the experience index by 17.7–23.6% without any drop in task performance.
- Structure Over Geometry: The paper argues that for cross-scene transfer, the topology (how rooms connect) is far more valuable than the geometry (exact coordinates or distances).
- Frozen VLMs are "Memory-Limited": The results show that we don't necessarily need to fine-tune massive models for navigation. Instead, providing them with structured, relevant "memories" of past successes is enough to significantly boost their reasoning capabilities.
Key Results
HTCE outperformed all baselines, including those using textual memories or specialized navigation policies:
- Success Rate (SR): Achieved 59.9% on HM3D v0.1 and 71.0% on HM3D v0.2, representing gains of up to 4.9 points over the strongest baselines.
- Efficiency (SPL): Structured retrieval contributed up to 2.8 points to the Success weighted by Path Length (SPL), proving that the agent wasn't just finding targets more often, but finding them more efficiently.
- Robustness: The system maintained its lead across different datasets using the exact same frozen backbone and configuration.
Qualitative comparison showing HTCE reaching a target efficiently while a baseline (WMNav) continues to explore aimlessly. (Note: Figure 3 in the paper provides a visual walkthrough of this success vs. failure case).
Limitations and Future Work
- Manual Role Mapping: Currently, a human must define which labels count as "transitions" vs. "anchors." Future work could involve the VLM automatically determining which nodes are topologically significant.
- Residential Focus: The current transition vocabulary is tailored for homes. Applying HTCE to offices or outdoor urban environments would require re-tuning the semantic role map.
- Dynamic Environments: The framework assumes a relatively static environment where the "bones" of the house don't change during an episode.
Overall, HTCE provides a mathematically grounded way to make robot "experience" truly transferable, moving us closer to agents that can walk into any home and know exactly where the kitchen is.
Summary updated