EO-Agents: A Three-Agent LLM Pipeline for Earth Observation Hypothesis Generation

Mahyar Ghazanfari, Amin Tabrizian, Armin Mehrabian, Peng WeiAI4Science @ ICML 2026 · OpenReview

TL;DR

Researchers have developed EO-Agents, a pipeline that moves beyond vague, text-based AI suggestions to generate concrete, data-grounded Earth science hypotheses. By combining a Graph Neural Network (GNN) with a three-agent LLM team, the system identifies novel pairings of NASA datasets and produces research plans that are nearly as plausible as those found in published literature.

What Problem Does This Paper Address?

Scientific discovery in Earth Observation (EO) is fundamentally a "matching" problem. With over 8,000 NASA datasets spanning five decades, the number of possible ways to combine measurements—like pairing soil moisture from one satellite with vegetation indices from another—is astronomical (roughly $10^6$ pairings).

While Large Language Models (LLMs) are great at generating ideas, they usually rely on unstructured text. This leads to "hallucinated" or overly vague suggestions like "use satellite data to study drought." In the real world, a researcher needs to know exactly which data products to use. The authors address this "grounding" gap by forcing the AI to work directly with a structured knowledge graph of actual NASA measurement products.

Methods

The EO-Agents pipeline consists of two main stages:

  1. The Structural Ranker: The team built a Heterogeneous Graph Neural Network (GNN). Unlike a standard neural net, this model understands the relationships between different types of nodes (Datasets, Platforms, Instruments, etc.). It was trained on historical "co-usage" data—essentially learning which datasets have been cited together in past papers—to predict which currently unpaired datasets should be combined.
  2. The Three-Agent LLM Cascade: The top 200 pairings from the GNN are passed to a trio of LLM agents:
  3. Agent 1 (Filter): Rescores pairs based on scientific plausibility and novelty.
  4. Agent 2 (Generator): Articulates a structured research hypothesis, including a testable claim, analysis method, and expected findings.
  5. Agent 3 (Judge): Evaluates the final hypothesis on importance, tractability, and novelty.

To ensure the results weren't just a fluke of one specific model, the authors ran a $2 \times 2 \times 2$ factorial experiment, swapping GPT-5.2 and Claude Sonnet 4.6 into different agent roles to see how the "identity" of the AI affected the quality of the science.

Datasets

  • NASA Earth Observation Knowledge Graph (EO-KG): A massive resource containing 150,351 nodes and 436,203 edges. It includes metadata for 1,475 primary datasets.
  • SPECTER2 Embeddings: Used to provide the GNN with a "semantic" understanding of dataset abstracts.
  • Co-usage Supervision: A derived dataset of 23,490 unique pairs of datasets that have appeared together in the same publication, used to train and test the GNN ranker.

Key Insights and Ideas

  • Data-Grounding vs. Text-Grounding: By pinning every hypothesis to two specific NASA datasets by construction, the pipeline ensures that every idea is "actionable" today using public archives.
  • The "Judge" Bias: One of the most striking findings was that while different LLMs (GPT vs. Claude) generally agreed on which hypotheses were better than others (stable ranking), their absolute scores varied wildly. GPT tended to be a "lenient grader," scoring importance ~0.6 points higher than Claude across the board.
  • Context Matters for Tractability: When the "Judge" agent was shown the actual dataset metadata (the "Contextual" condition) rather than just the hypothesis text ("Blind"), tractability scores dropped. This suggests that LLMs realize a project is harder to execute once they see the "fine print" of the data products involved.
  • GNNs Beat Simple Search: The heterogeneous GNN outperformed standard text-similarity baselines, particularly in finding "cross-archive" pairings (e.g., combining data from an atmospheric archive with data from an oceanography archive).

Key Results

  • High Plausibility: LLM judges rated the GNN-predicted novel pairings at 4.38/5 for plausibility, nearly identical to the 4.57/5 given to real-world pairings from 2024 publications.
  • Diverse Output: The system produced 160 structured hypotheses across domains like glaciology, ecohydrology, and aerosol-cloud interactions.
  • GNN Performance: The GNN ranker achieved a Hits@10 of 0.468, which is 4.7x better than random and significantly higher than popularity-based or simple semantic-search baselines.
  • Stable Rankings: Despite the scoring bias mentioned above, 159 out of 160 hypotheses maintained stable relative rankings regardless of which model was judging, suggesting that LLM-as-a-judge is reliable for filtering, if not for absolute "grading."

Limitations and Future Work

  • Human Verification: The 160 hypotheses are currently LLM-judged. The authors acknowledge that a domain-expert study is the necessary next step to prove these "seeds" can grow into real-world discoveries.
  • Abstract Truncation: To fit within LLM context windows, dataset abstracts were truncated to 1,200 characters, which might omit technical details crucial for some complex pairings.
  • Ideation, Not Execution: The current pipeline suggests what to do, but it doesn't yet have "tool use" capabilities to actually download the data and run the code. Integrating an autonomous "executor" agent is a clear future direction.

Summary updated

Abstract

Large language models have recently been explored for scientific hypothesis generation, but most prior work relies on unstructured literature and free-form textual claims. We present a pipeline for Earth observation that grounds hypothesis generation directly in the NASA Earth Observation Knowledge Graph. A heterogeneous graph neural network trained on historical co-usage relations ranks candidate dataset pairings, and a three-agent LLM pipeline filters, generates, and evaluates structured research hypotheses. Applied to 1,475 NASA datasets, the system produces 160 hypotheses spanning multiple Earth-science domains, including ecohydrology, glaciology, aerosol--cloud interactions, vegetation phenology, and stratospheric chemistry. Model-predicted novel dataset pairings are rated nearly as plausible as held-out real co-usages from the literature, indicating that the pipeline surfaces scientifically coherent yet unexplored combinations. A $2 \times 2 \times 2$ factorial experiment across GPT-5.2 and Claude Sonnet 4.6 shows that hypothesis rankings remain stable, while absolute scores depend strongly on judge identity, highlighting limitations of single-judge LLM evaluation.
BibTeX
@inproceedings{ghazanfari2026eo,
  author = {Mahyar Ghazanfari and Amin Tabrizian and Armin Mehrabian and Peng Wei},
  title = {{EO-Agents: A Three-Agent LLM Pipeline for Earth Observation Hypothesis Generation}},
  booktitle = {AI4Science @ ICML},
  year = {2026},
  url = {https://openreview.net/forum?id=mqlvHLejoG},
}
Figure