TL;DR
HSGNav introduces the Hypothesis Scene Graph (HSG), a dual-layer memory structure that separates stable 3D furniture "entities" from revisable surface-object "hypotheses." This allows assistive robots to efficiently track and relocate tiny items (like an apple) across long-term deployments without the brittle requirement of building full 3D geometry for every small object.
What Problem Does This Paper Address?
Imagine asking a robot to "find my glasses." In a typical home, that robot might have seen them on the coffee table an hour ago. But if you moved them to the nightstand, most current navigation systems fail in one of two ways:
- Stale Memory: They keep searching the coffee table because their internal 3D map says the glasses are part of that location's geometry.
- Scale Insensitivity: Small objects often produce noisy depth data or intermittent detections, causing them to be filtered out of 3D maps entirely or incorrectly merged into the furniture they sit on.
Existing "lifelong" navigation benchmarks often focus on static scenes or large furniture. HSGNav tackles the reality of multi-scale navigation under object relocation, where robots must find everything from a couch to a coin, even as the environment changes.
Methods
The core innovation is the Hypothesis Scene Graph (HSG). Instead of a "flat" graph where every detection is a 3D node, HSG creates two levels:
- Entity Level: Persistent 3D furniture (tables, counters) that serve as spatial anchors.
- Hypothesis Level: Revisable records of small objects found on those anchors. These don't require independent 3D geometry; they are essentially "visual evidence" linked to a piece of furniture.
Building on this structure, the HSGNav system employs three key mechanisms:
- Scale-Adaptive Memory Retrieval (SAMR): Uses a Vision-Language Model (VLM) to intelligently filter the graph. If you're looking for a "refrigerator," it looks at entities; if you're looking for a "lemon," it focuses on surface hypotheses.
- Belief-Driven Verification (BDV): When the robot reaches a remembered location and the target is missing, it doesn't just give up or rebuild the whole map. It locally invalidates that specific hypothesis, instantly cleaning its memory.
- Observation-Complete Exploration (OCE): If the target isn't in memory, the robot doesn't just wander into new rooms. It prioritizes "under-observed" surfaces—checking the tops of tables it has passed but hasn't looked at closely yet.
Datasets
The authors evaluate HSGNav on:
- GOAT-Bench: A standard benchmark for lifelong navigation in static scenes.
- FAROS-Bench (Find And Re-find Objects across Scales): A new, more challenging benchmark introduced in this paper. It features 210 subtasks where objects of varying scales (tiny, small, large) are relocated mid-episode, forcing the robot to update its memory to succeed.
Key Insights and Ideas
- Decoupling Geometry from Evidence: The "aha moment" is realizing that for small objects, we don't need a perfect 3D mesh; we just need to know which surface they were seen on and what they looked like. This prevents the "fusion" errors that plague traditional 3D scene graphs.
- Local vs. Global Revision: By anchoring hypotheses to stable furniture, the robot can "delete" a moved object from its mind without affecting its understanding of the room's layout.
- Surface-First Exploration: In a home, small objects are almost always on surfaces. Prioritizing the inspection of tabletops over exploring distant hallways significantly boosts the success rate for finding "tiny" targets.
Key Results
HSGNav sets a new state-of-the-art (SOTA) on both benchmarks:
- GOAT-Bench: Achieved a 71.2% Success Rate (SR), outperforming the previous best (3D-Mem) by 2.1%.
- FAROS-Bench: The performance gap was even wider here. HSGNav achieved an overall 58.1% SR, nearly doubling the performance of baselines on "dynamic" tasks where objects had been moved.
- Efficiency: HSGNav is significantly leaner, requiring roughly half the VLM calls and one-sixth the images compared to 3D-Mem, making it more viable for real-time robotics.
Limitations and Future Work
The current system relies on stable furniture anchors. If a person moves a chair or a table, the spatial reference for the hypotheses attached to it might break. The authors suggest that a natural next step is to develop methods to maintain and reassociate these hypotheses even when the "anchor" entities themselves are relocated. Additionally, while the system uses VLMs effectively, the computational cost of frequent VLM calls remains a hurdle for low-power mobile robots.
Summary updated