DocFS: File Systems as Agentic Reasoning Environments for Complex Multimodal Document Question Answering

Bingdian Zhang, Zixuan Li, Long Bai, Xiaolong Jin, Jiafeng Guo, Xueqi ChengarXiv 2026 · arXiv

TL;DR

DocFS transforms complex, multi-page documents into hierarchical file systems, allowing AI agents to navigate sections and access multimodal content using standard tools like ls, grep, and read_file. This approach achieves state-of-the-art performance on long-document benchmarks while being up to 10.3x faster and using 10.1x fewer tokens than previous top-tier methods.

What Problem Does This Paper Address?

Multimodal Document Question Answering (DocQA) is a significant challenge for Vision Language Model (VLM) agents. Real-world documents—like financial reports or academic papers—are long, hierarchical, and contain information scattered across text, tables, figures, and equations.

Current approaches generally fall into two camps:

  1. Flat Organization: Treating the document as a sequence of pages. This loses the structural context (e.g., knowing a figure belongs to "Section 3.2").
  2. Custom Structured Representations: Building complex graphs or trees. While these capture structure, they often lack intuitive, standardized tools for an agent to actually explore that structure efficiently.

The authors identify a gap: we need an environment that not only represents document structure but also provides a set of "aligned tools" that allow an agent to navigate, search, and verify information just like a human researcher would.

Methods

The core innovation is DocFS, which treats a document as a virtual operating system environment. The process follows three main stages:

  1. Document Preprocessing: Raw PDFs are parsed into multimodal elements. Crucially, the authors use an LLM to reconstruct the section hierarchy (fixing "flattened" headers) and a VLM to correct errors in parsed equations and tables.
  2. File System Construction: The document is reorganized into a directory tree.
  3. Directories mirror the section hierarchy.
  4. Files store content in native formats (Markdown for text, CSV for tables, JPG for images).
  5. Metadata Files: index.md files provide navigation entry points, while relation.json files explicitly link text to relevant figures or tables.
  6. Image Descriptions: VLMs generate textual summaries of visual elements, making them searchable via keyword search.
  7. Agentic Reasoning: Instead of a specialized API, the agent uses standard file system tools:
  8. ls: To see what sections and assets are available.
  9. grep: For global keyword search across the entire "document-drive."
    • read_file: To examine specific content.
  10. glob: To collect files of a specific type (e.g., all figures).
Figure 1: An overview of the agentic framework of DocFS.

Datasets

The authors evaluated DocFS on two rigorous benchmarks:

  • MMLongBench-Doc: A dataset of lengthy, multi-domain documents requiring models to integrate evidence across different regions and modalities.
  • FinRAGBench-V: A benchmark focused on financial reports with dense, complex layouts and heavy reliance on tables and charts.

Key Insights and Ideas

  • The File System Metaphor Works: File systems are a mature solution for organizing hierarchical data. By mapping documents to this structure, agents can use well-understood logic (navigation and search) to solve QA tasks.
  • Tool-Structure Alignment: The "aha moment" is that an agent's effectiveness depends on its tools. A structured document is useless if the agent can't "walk" through it. Standard OS tools provide a flexible, robust interface for this exploration.
  • Searchable Visuals: By generating textual descriptions for images and storing them as searchable files, DocFS allows agents to "find" a figure via grep before "looking" at it with a VLM.
  • Efficiency through Localization: Because the agent can navigate directly to a relevant "folder" (section), it avoids processing irrelevant pages, leading to massive savings in time and compute costs.

Key Results

DocFS set new state-of-the-art records across both benchmarks using backbone models like Qwen3.5-Plus and DeepSeek-V4.1-Flash.

  • Accuracy: On MMLongBench-Doc, DocFS achieved 71.6% accuracy, outperforming the strongest training-free baseline (DocLens) by 2.9 percentage points. On FinRAGBench-V, it reached 82.3%, a 4.3-point lead.
  • Speed and Cost: The efficiency gains were dramatic. As shown in the table below, DocFS is significantly faster and more token-efficient. For instance, using DeepSeek-V4.1-Flash on financial reports, DocFS was 10.3x faster and used 10.1x fewer tokens than DocLens.
Table 3: Efficiency comparison between DocLens and DocFS. Abbreviations: Qwen-Plus = Qwen3.5-Plus, DS-Flash = DeepSeek-V4.1-Flash.

Ablation studies confirmed that grep (keyword search) and ls (directory listing) are the most critical tools; removing grep caused the largest performance drop, as agents were forced to "guess" which files contained the answers.

Limitations and Future Work

While DocFS is highly effective, the authors note several areas for growth:

  • Adaptive Organization: The current hierarchy is fixed during preprocessing. Future versions could allow agents to reorganize the "file system" dynamically based on the specific question.
  • Complex Cross-Modal Reasoning: While relation.json helps, navigating extremely dense associations between hundreds of entities across modalities remains a challenge.
  • Cost-Reliability Balance: There is still a trade-off between how much an agent explores (cost) and the final answer's accuracy. Optimizing this "stopping criteria" is a promising research direction.

Summary updated

Abstract

VLM-based agents solve complex tasks by iteratively interacting with environments, and their effectiveness depends heavily on how the environment exposes information and provides tools for acting on it. However, constructing an effective environment for complex multimodal document question answering remains challenging: documents usually span many pages, with multimodal contents distributed across hierarchical sections and linked by cross-modal associations, while existing methods organize documents into structured representations that address only the information side of the environment and lack tools aligned with the structure, limiting agents' ability to fully exploit it. We observe that file systems naturally combine structure organization with aligned tools: directories and files encode document structure, while file operations provide rich tools to access file contents. Building on this insight, we propose DocFS, a file-system-based agentic framework that reorganizes raw documents as file systems, where directories mirror the section hierarchy and explicit links connect text with figures, tables, and equations. Based on well-established file system tools such as navigation, search, and reading, agents can iteratively explore the document structure, locate relevant evidence, and understand fine-grained local information. Paired with Qwen3.5-Plus, DocFS achieves state-of-the-art performance on MMLongBench-Doc and FinRAGBench-V, outperforming the strongest baselines by 2.9 and 4.3 percentage points, respectively, and is up to 10.3x faster with up to 10.1x fewer tokens than the strongest baseline during reasoning.
BibTeX
@misc{zhang2026docfs,
  author = {Bingdian Zhang and Zixuan Li and Long Bai and Xiaolong Jin and Jiafeng Guo and Xueqi Cheng},
  title = {{DocFS: File Systems as Agentic Reasoning Environments for Complex Multimodal Document Question Answering}},
  year = {2026},
  eprint = {2611.12957},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12957},
}
Figure