TL;DR
EviSeek is a training-free agentic framework that solves "undirected interaction" in embodied AI by making information needs explicit. By decomposing questions into specific "evidence demands" and selectively retaining only relevant observations on a structured "evidence board," EviSeek significantly improves both accuracy and token efficiency across multiple vision-language model (VLM) backbones.
What Problem Does This Paper Address?
In Active Embodied Question Answering (A-EQA), an agent must move through a 3D environment and interact with objects to answer a question (e.g., "Which bottle is larger?"). The central challenge is undirected interaction: agents often take plausible-looking actions (like moving toward a table) without a clear plan for what specific evidence those actions should reveal.
Existing agents often suffer from "action blindness," where they fail to acquire necessary observations or discard crucial evidence from their limited visual history before reaching the final answer. EviSeek addresses this by shifting the focus from "what action should I take next?" to "what evidence do I need to find?"
Methods
EviSeek is a training-free framework that uses prompted VLM operators to manage the information-seeking process. It consists of three main modules:
- Evidence-Demand Planning: Upon receiving a question and an initial view, the agent generates a set of evidence demands ($D$). These are specific, observable targets (e.g., "The size of Bottle A") that are necessary to answer the question. It then creates an acquisition plan consisting of subtasks assigned to these demands.
- Evidence-Seeking Interaction: The agent pursues these subtasks through viewpoint changes or physical interventions. Crucially, it uses a "seeker" policy to select actions and a "curator" policy to decide if a new observation actually satisfies a demand.
- Evidence-Oriented Memory: Instead of a simple sliding window of recent frames, EviSeek maintains an evidence board. This board selectively retains only the observations and action outcomes that were flagged as relevant by the curator, linked directly to their originating demand.
Datasets
The authors evaluated EviSeek on two major benchmarks:
- ESI-Bench: A large-scale benchmark containing 3,436 questions across 10 task families (e.g., spatial relations, physical dynamics, cognitive mapping) in the OmniGibson simulator.
- EXPRESS-Bench: A navigation-focused benchmark used to test if the framework's benefits extend to finding information in larger, multi-room environments.
Key Insights and Ideas
- Explicit Demands Reduce Noise: By defining what evidence is needed upfront, the agent avoids getting lost in irrelevant environmental details. Ablation studies showed that removing these explicit demands dropped accuracy by 6.1 percentage points.
- Selective Retention is Efficient: Storing only "provenance-linked" evidence on the board, rather than a raw history, provides a much cleaner context for the final answering stage. This allows the agent to use fewer images at answer time while achieving higher accuracy.
- Training-Free Flexibility: Because EviSeek relies on prompting rather than fine-tuning, it is backbone-agnostic. It showed consistent improvements when using both open-weight (Qwen3-VL-32B) and closed (Gemini 3.1 Pro) models.
- Interaction as Information Gathering: The framework treats physical actions (like picking up an object) not just as tasks to be completed, but as experiments to reveal hidden information.
Key Results
EviSeek demonstrated substantial gains over standard "Single-model" active agents:
- Accuracy Boost: On ESI-Bench, EviSeek improved accuracy by 7.1% with Qwen3-VL-32B and 3.7% with Gemini 3.1 Pro.
- Resource Efficiency: Despite having multiple VLM stages, EviSeek reduced mean token usage per episode by 45.1% for Qwen and 13.9% for Gemini. It also required fewer model calls on average.
- Visual Support: The framework significantly increased "grounded correctness"—answers that are not only correct but are demonstrably supported by the acquired images.
Limitations and Future Work
- Spatial Reasoning Bottleneck: EviSeek does not build a metric spatial map; it relies entirely on the VLM's internal spatial reasoning. This led to regressions in the "Metric Comparison" category (e.g., judging precise distances).
- Sim-to-Real Gap: The framework was tested only in simulated environments. Real-world factors like sensor noise and uncertain physical contact might challenge the "curator's" ability to verify evidence.
- Future Directions: The authors suggest that integrating explicit spatial modeling or using even stronger native spatial reasoning models (like the upcoming GPT-6 Astra) could further enhance the framework.
Summary updated