Introduction
Instance ImageGoal Navigation (IIN) is a challenging task in embodied AI where an agent must locate a specific object instance within a complex environment, guided only by a goal image. Unlike standard ImageNav, which targets object categories, IIN requires distinguishing the target from visually similar distractors—for example, finding a specific bed with unique pillows among several beds in a house. Existing approaches often struggle with this ambiguity, frequently stopping at the first similar-looking object they encounter.
The IEVE Framework
To address these challenges, the Instance-aware Exploration-Verification-Exploitation (IEVE) framework introduces a dedicated "Verification" phase between exploration and exploitation. This mimics the human behavior of approaching a potential target to confirm its identity before committing to it as the goal.
The framework consists of five modular components:
- Instance Classification: Determines the target's category from the goal image.
- Online Mapping: Constructs a real-time 2D semantic map using RGB-D data and instance segmentation (e.g., InternImage).
- Switch Policy: The core logic that toggles between Exploration (searching for the category), Verification (approaching a candidate), and Exploitation (navigating to a confirmed instance).
- Goal Mapping Policy: Sets long-term coordinates based on the current state.
- Local Policy: Executes low-level navigation to reach the chosen coordinates.
Distance-Aware Verification
The "Verification" step utilizes local feature matching (DISK and LightGlue) to compare the agent's current view with the goal image. A critical innovation is the use of an adaptive threshold for confirmation. Because visual matching reliability is distance-dependent, the agent requires more stringent matching criteria when far away. As the agent moves closer, the threshold relaxes, allowing for a robust confirmation process that effectively filters out distractors.
Results and Discussion
Evaluated on the HM3D-SEM dataset within the Habitat simulator, IEVE achieves a state-of-the-art success rate of 70.2%, significantly outperforming the previous best of 56.1%. Notably, the framework demonstrates extreme efficiency compared to end-to-end reinforcement learning methods, which achieve less than 10% success even after billions of training steps.
While IEVE is highly effective, it remains dependent on the accuracy of the underlying segmentation models. Future research will explore incorporating high-level semantic reasoning—such as understanding that beds are typically found in bedrooms—to further optimize the exploration phase and adapt the system for dynamic real-world environments.
Summary updated