Knowledge-Grounded Agentic Large Language Models for Multi-Hazard Understanding from Reconnaissance Reports

Chenchen Kuai, Zihao Li, Braden Rosen, Stephanie Paal, Navid Jafari, Jean-Louis Briaud, Yunlong Zhang, Youssef M. A. Hashash, Yang ZhouarXiv 2025 · DOI · arXiv

Introduction

Post-disaster reconnaissance reports are a critical resource for engineers and emergency managers, documenting everything from soil liquefaction to specific structural failure modes. However, these reports are often hundreds of pages of unstructured, technical text. While Large Language Models (LLMs) are theoretically suited for this data, they often "hallucinate" or lose context when dealing with complex, cascading hazards.

To address this, researchers from Texas A&M and the University of Illinois introduced MoRA-RAG (Mixture-of-Retrieval Agentic RAG). This framework is designed to transform unstructured disaster narratives into a reliable foundation for multi-hazard reasoning, achieving high accuracy in complex engineering domains.

The MoRA-RAG Framework

MoRA-RAG moves beyond "Vanilla RAG" through three core innovations:

  • Agentic Chunking: An LLM agent identifies standalone factual "propositions" and groups them into semantically coherent chunks, preserving the logical flow of engineering observations.
  • Mixture-of-Retrieval (MoR): A router agent determines which hazard-specific databases are relevant to a query and dynamically allocates a retrieval budget to capture cascading effects (e.g., an earthquake triggering a landslide).
  • The Agentic Loop: A five-agent pipeline—comprising a Router, Evaluator, Online Search, Rewriter, and Answer Writer—iteratively verifies evidence sufficiency before synthesizing a grounded response.

The HazardRecQA Dataset

The framework was evaluated using HazardRecQA, a new benchmark derived from Geotechnical Extreme Events Reconnaissance (GEER) reports. This dataset encompasses 5,776 question-answer pairs across 90 global disaster events, providing a robust testbed for multi-hazard understanding.

Global distribution of disaster events and hazard types included in the HazardRecQA benchmark.
Distribution of hazard types within the HazardRecQA dataset, showing the quantitative breakdown of the 5,776 question-answer pairs.

Key Results and Insights

MoRA-RAG achieved 94.5% accuracy, significantly outperforming zero-shot LLMs and existing RAG systems. Key insights from the study include:

  • Hallucination Reduction: The "Evidence Evaluator" agent allows the system to recognize when information is missing, preventing forced, incorrect answers.
  • Model Democratization: MoRA-RAG enables mid-sized, open-weight models (12B+ parameters) to perform on par with proprietary models, facilitating reproducible and locally-hosted research.
  • Query Stability: The system maintained consistent performance across various question categories, including hazard characteristics, impact analysis, and response recovery.

Conclusion

While the multi-agent approach increases latency (averaging ~20 seconds per query), the gains in accuracy and reliability are vital for high-stakes disaster science. Future work will explore multi-modal integration, allowing the framework to correlate textual reports with satellite imagery and sensor data to provide a more holistic understanding of disaster impacts.

Summary updated

Abstract

Post-disaster reconnaissance reports contain critical evidence for understanding multi-hazard interactions, yet their unstructured narratives make systematic knowledge transfer difficult. Large language models (LLMs) offer new potential for analyzing these reports, but often generate unreliable or hallucinated outputs when domain grounding is absent. This study introduces the Mixture-of-Retrieval Agentic RAG (MoRA-RAG), a knowledge-grounded LLM framework that transforms reconnaissance reports into a structured foundation for multi-hazard reasoning. The framework integrates a Mixture-of-Retrieval mechanism that dynamically routes queries across hazard-specific databases while using agentic chunking to preserve contextual coherence during retrieval. It also includes a verification loop that assesses evidence sufficiency, refines queries, and initiates targeted searches when information remains incomplete. We construct HazardRecQA by deriving question-answer pairs from GEER reconnaissance reports, which document 90 global events across seven major hazard types. MoRA-RAG achieves up to 94.5 percent accuracy, outperforming zero-shot LLMs by 30 percent and state-of-the-art RAG systems by 10 percent, while reducing hallucinations across diverse LLM architectures. MoRA-RAG also enables open-weight LLMs to achieve performance comparable to proprietary models. It establishes a new paradigm for transforming post-disaster documentation into actionable, trustworthy intelligence for hazard resilience.
BibTeX
@misc{kuai2025knowledge,
  author = {Chenchen Kuai and Zihao Li and Braden Rosen and Stephanie Paal and Navid Jafari and Jean-Louis Briaud and Yunlong Zhang and Youssef M. A. Hashash and Yang Zhou},
  title = {{Knowledge-Grounded Agentic Large Language Models for Multi-Hazard Understanding from Reconnaissance Reports}},
  year = {2025},
  eprint = {2511.14010},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.14010},
}
Figure