EviSeek: An Evidence-Centric Agentic Framework for Embodied Active Information Seeking

Jintao Li, Zhiyong Cao, Baihua Xiao, Yu Bai, Börje F. KarlssonarXiv 2026 · arXiv

TL;DR

EviSeek is a training-free agentic framework that solves "undirected interaction" in embodied AI by making information needs explicit. By decomposing questions into specific "evidence demands" and selectively retaining only relevant observations on a structured "evidence board," EviSeek significantly improves both accuracy and token efficiency across multiple vision-language model (VLM) backbones.

What Problem Does This Paper Address?

In Active Embodied Question Answering (A-EQA), an agent must move through a 3D environment and interact with objects to answer a question (e.g., "Which bottle is larger?"). The central challenge is undirected interaction: agents often take plausible-looking actions (like moving toward a table) without a clear plan for what specific evidence those actions should reveal.

Existing agents often suffer from "action blindness," where they fail to acquire necessary observations or discard crucial evidence from their limited visual history before reaching the final answer. EviSeek addresses this by shifting the focus from "what action should I take next?" to "what evidence do I need to find?"

Methods

EviSeek is a training-free framework that uses prompted VLM operators to manage the information-seeking process. It consists of three main modules:

  1. Evidence-Demand Planning: Upon receiving a question and an initial view, the agent generates a set of evidence demands ($D$). These are specific, observable targets (e.g., "The size of Bottle A") that are necessary to answer the question. It then creates an acquisition plan consisting of subtasks assigned to these demands.
  2. Evidence-Seeking Interaction: The agent pursues these subtasks through viewpoint changes or physical interventions. Crucially, it uses a "seeker" policy to select actions and a "curator" policy to decide if a new observation actually satisfies a demand.
  3. Evidence-Oriented Memory: Instead of a simple sliding window of recent frames, EviSeek maintains an evidence board. This board selectively retains only the observations and action outcomes that were flagged as relevant by the curator, linked directly to their originating demand.
Figure 1: EviSeek connects question-derived evidence demands to acquisition goals, observation-conditioned actions, and retained evidence for answering. Unselected observations support control but are excluded from the evidence board.

Datasets

The authors evaluated EviSeek on two major benchmarks:

  • ESI-Bench: A large-scale benchmark containing 3,436 questions across 10 task families (e.g., spatial relations, physical dynamics, cognitive mapping) in the OmniGibson simulator.
  • EXPRESS-Bench: A navigation-focused benchmark used to test if the framework's benefits extend to finding information in larger, multi-room environments.

Key Insights and Ideas

  • Explicit Demands Reduce Noise: By defining what evidence is needed upfront, the agent avoids getting lost in irrelevant environmental details. Ablation studies showed that removing these explicit demands dropped accuracy by 6.1 percentage points.
  • Selective Retention is Efficient: Storing only "provenance-linked" evidence on the board, rather than a raw history, provides a much cleaner context for the final answering stage. This allows the agent to use fewer images at answer time while achieving higher accuracy.
  • Training-Free Flexibility: Because EviSeek relies on prompting rather than fine-tuning, it is backbone-agnostic. It showed consistent improvements when using both open-weight (Qwen3-VL-32B) and closed (Gemini 3.1 Pro) models.
  • Interaction as Information Gathering: The framework treats physical actions (like picking up an object) not just as tasks to be completed, but as experiments to reveal hidden information.

Key Results

EviSeek demonstrated substantial gains over standard "Single-model" active agents:

  • Accuracy Boost: On ESI-Bench, EviSeek improved accuracy by 7.1% with Qwen3-VL-32B and 3.7% with Gemini 3.1 Pro.
  • Resource Efficiency: Despite having multiple VLM stages, EviSeek reduced mean token usage per episode by 45.1% for Qwen and 13.9% for Gemini. It also required fewer model calls on average.
  • Visual Support: The framework significantly increased "grounded correctness"—answers that are not only correct but are demonstrably supported by the acquired images.
Accuracy breakdown across ESI-Bench categories, showing EviSeek's consistent performance gains.
Table 2: Mean resource use across all 3,436 ESI-Bench questions: primitive steps, VLM calls, and total tokens per episode. Images denotes images per answer.

Limitations and Future Work

  • Spatial Reasoning Bottleneck: EviSeek does not build a metric spatial map; it relies entirely on the VLM's internal spatial reasoning. This led to regressions in the "Metric Comparison" category (e.g., judging precise distances).
  • Sim-to-Real Gap: The framework was tested only in simulated environments. Real-world factors like sensor noise and uncertain physical contact might challenge the "curator's" ability to verify evidence.
  • Future Directions: The authors suggest that integrating explicit spatial modeling or using even stronger native spatial reasoning models (like the upcoming GPT-6 Astra) could further enhance the framework.

Summary updated

Abstract

Active embodied question answering requires agents to acquire evidence through viewpoint changes and physical interaction. The central challenge is to avoid undirected interaction: taking plausible actions without specifying what evidence they should reveal for answering the question. Action plans and accumulated histories alone can leave the informational purpose of an interaction implicit. We introduce EviSeek, a training-free framework that makes question-derived evidence demands explicit acquisition objectives, linking observation-conditioned actions and selectively retained evidence to the information the agent seeks. On 3,436 ESI-Bench questions, EviSeek improves accuracy over the Single-model agent by 7.1 percentage points with Qwen3-VL-32B and 3.7 points with Gemini 3.1 Pro, while improving visual-support-weighted accuracy and reducing mean token usage. These findings support making the informational purpose of interaction explicit, with action selection and evidence retention organized around what the agent needs to learn to answer the question.
BibTeX
@misc{li2026eviseek,
  author = {Jintao Li and Zhiyong Cao and Baihua Xiao and Yu Bai and Börje F. Karlsson},
  title = {{EviSeek: An Evidence-Centric Agentic Framework for Embodied Active Information Seeking}},
  year = {2026},
  eprint = {2611.06434},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.06434},
}
Figure