TL;DR
AVLEN is a multimodal embodied navigation agent that combines audio-visual cues with the ability to interactively query a human "oracle" for help. Using a hierarchical reinforcement learning framework, the agent learns to autonomously navigate 3D environments and strategically ask for natural language instructions when it encounters uncertainty, significantly improving success rates in complex, noisy scenarios.
What Problem Does This Paper Address?
Imagine a robot tasked with finding a "thud" in a large house. In traditional AI research, this problem is usually split into two silos: Vision-and-Language Navigation (VLN), where agents follow a set of instructions but are "deaf," and Audio-Visual Navigation (AVN), where agents hunt for a sound source but cannot understand language.
In the real world, sounds are often sporadic (a door slams once and then it is quiet) or ambiguous (multiple appliances running at once). Current agents often get "lost" in these scenarios. While some previous works allowed agents to ask for help, they relied on rigid heuristics—like asking for help only when a probability score dropped below a certain threshold. The authors of AVLEN argue that an agent should learn the value of a human's advice and decide for itself when the cost of a query is worth the navigation boost.
Methods
The core of the paper is a Multimodal Hierarchical Reinforcement Learning framework. Instead of one giant neural network, AVLEN uses a two-level hierarchy to manage its navigation and querying behavior:
- High-Level Policy ($\pi_q$): This acts as the "manager." At each step, it looks at the agent’s history and current audio-visual state to decide: "Should I keep navigating on my own, or should I ask the oracle for help?" It is trained with a reward function that balances the desire to reach the goal with a penalty for "bothering" the oracle too much.
- Low-Level Policies:
- Audio-Visual Policy ($\pi_g$): Navigates based on RGB-D images and binaural (stereo) audio. It uses a Transformer-based architecture to remember past observations.
- Language Policy ($\pi_l$): If the manager decides to ask for help, the oracle provides a short instruction (e.g., "Go towards the dining table"). This policy uses CLIP embeddings and a Transformer to translate that text into a sequence of physical actions.
To train this system without a human sitting at a computer for millions of iterations, the authors used a "Speaker" model—a separate AI trained to generate human-like instructions based on the shortest path to the goal.
Datasets
The researchers evaluated AVLEN using the SoundSpaces framework, which is built on top of Matterport3D.
- Matterport3D: Provides 85 highly detailed, real-world 3D scans of houses.
- SoundSpaces: Simulates how sound actually travels through those specific rooms, accounting for reverberation and occlusion by walls.
- SAVi (Semantic Audio-Visual Navigation): A benchmark where the agent must find an object (like a cat or a telephone) that makes a sound sporadically.
Key Insights and Ideas
- Learning Uncertainty: Rather than using a hard-coded threshold to trigger a help request, AVLEN treats "asking for help" as just another action in a Reinforcement Learning setting. This allows the agent to implicitly learn the "uncertainty" of its own audio-visual sensors.
- Judicial Querying: The agent doesn't just spam the oracle at the start. The results show it learns to "save" its queries for when the sound stops or when it enters a confusing part of the house.
- Language as a Shortcut: Natural language provides high-level semantic information that raw audio cannot. A single instruction like "turn left at the sofa" can resolve ambiguity that might take an audio-only agent minutes of wandering to figure out.
Key Results
The authors tested AVLEN in three tough scenarios: sporadic sounds, unheard sounds (objects not seen during training), and distractor sounds (background noise).
- Performance Boost: In the "unheard sound" setting, AVLEN achieved a 26.2% success rate, outperforming the state-of-the-art SAVi baseline (24.8%).
- The "Oracle" Gap: When the agent was given ground-truth actions instead of language instructions, the success rate jumped to 36.7%. This suggests that the agent is very good at knowing when to ask, but still has room to improve at understanding the language it receives.
- Robustness to Noise: In environments with distractor sounds, AVLEN's ability to ask for help became even more valuable, showing a more significant lead over non-interactive baselines.
Limitations and Future Work
- Language Understanding: As noted by the "Oracle Gap," the current bottleneck is the language-to-action mapping. Future work could involve more sophisticated pre-trained language models.
- Oracle Realism: The simulated oracle always provides perfect, shortest-path instructions. Real humans might be vague, slow to respond, or even occasionally wrong.
- Dataset Diversity: The Matterport3D dataset is primarily composed of North American homes. The agent's "common sense" about where a dining table is located might not translate to different cultures or architectural styles.
- Language Constraint: The model is currently limited to English instructions.
Summary updated