LLM-Enabled Scientific Knowledge Diffusion Analysis

Uttam K. Rao, Madhav MaratheAAAI 2026 · DOI

TL;DR

Researchers at the University of Virginia have developed a multi-agent AI system that builds a "living" knowledge graph to track how scientific ideas spread. By combining specialized LLM agents with temporal modeling, the system can trace how a single strategic hire can transform a university’s research output, outperforming standalone models like GPT-4o in complex temporal reasoning and entity disambiguation.

What Problem Does This Paper Address?

Traditional "science of science" studies usually rely on static snapshots of citations and co-authorship. While useful, these snapshots miss the "how" and "when" of knowledge diffusion. For example, if a top AI researcher moves from Stanford to a mid-sized university, how long does it take for that expertise to "infect" other departments? How do mentorship lineages or patent-to-paper links drive innovation?

Existing bibliometric databases (like OpenAlex) often lack this temporal granularity, and standalone Large Language Models (LLMs) suffer from "hallucinations" or lack access to the very latest data. The authors address the challenge of building a system that can ingest messy, multi-source data and answer complex questions about the movement of expertise over time.

Methods

The authors introduce a multi-agent AI architecture built on the AutoGen framework. Instead of one giant model doing everything, they orchestrate five specialized roles:

  1. Ingestion Agents: Connect to APIs (OpenAlex, ORCID) and web scrapers to gather raw data.
  2. Extraction Agents: Use LLMs to turn unstructured text into structured "triples" (e.g., Researcher X mentored Student Y).
  3. Disambiguation Agents: Solve the "John Doe" problem by combining rule-based heuristics with LLM reasoning to ensure different researchers with the same name aren't merged.
  4. Integration Agents: Assemble the data into a Neo4j graph database, maintaining "provenance" (tracking exactly where every piece of info came from).
  5. Analysis Agents: Translate natural language questions (e.g., "How did University U gain expertise in cryptography?") into formal graph queries.

A standout feature is the use of Timed Automata. Each researcher is modeled as a stateful agent whose "state" (affiliation and expertise) changes over time based on events like new publications or moving to a new institution.

Datasets

The system integrates several massive, diverse datasets to create a holistic view of the scientific ecosystem:

  • OpenAlex: ~240M works and ~50M author entities for core bibliometrics.
  • ORCID: For self-reported researcher career timelines.
  • ROR (Research Organization Registry): To unify institutional names.
  • USPTO & PatCit: 10M patents and 40M patent-to-paper citation links to track technology transfer.
  • Web Scraping: For "genealogy" (advisor-student) data and recent news not yet in formal databases.

Key Insights and Ideas

  • The "Cascade Effect": The system can quantitatively model how a single "pivotal hire" acts as a catalyst. In one case study, they showed how one professor's arrival in 2004 led to a tenfold increase in a university's theoretical CS output within a decade through mentorship and subsequent hiring.
  • Provenance as a Cure for Hallucination: Unlike a standard chatbot, every answer this system provides is backed by a specific path in a graph database. If the AI says a researcher moved in 2019, it can point to the exact record that proves it.
  • Context-Aware Disambiguation: By using LLMs to look at a researcher's entire career trajectory and topic history, the system is much better at distinguishing between two different "John Does" than traditional algorithmic methods.

Key Results

The researchers tested their system against GPT-4o (with Deep Research) and Llama 3.1 8B on 20 complex queries:

  • Accuracy & Quality: The multi-agent system scored 4.3/5 on answer quality, beating GPT-4o (3.8) and Llama 3.1 (2.5).
  • Efficiency: While GPT-4o Deep Research took an average of 27 minutes to browse the web and synthesize an answer, this system responded in just 4.5 minutes.
  • Temporal Reasoning: Standalone LLMs often struggled with "before/after" logic or tracking career moves, whereas the graph-based approach handled these natively.
  • Data Recency: Because the ingestion agents can scrape the web in real-time, the system could answer questions about 2024 events that were past the "knowledge cutoff" of the base LLMs.

Limitations and Future Work

The authors acknowledge that there are currently no "gold-standard" benchmarks for these types of complex, multi-step scientific queries, making evaluation difficult.

Future directions include:

  • Fine-tuning: Developing domain-specific LLMs for even better extraction accuracy.
  • Cross-Language Support: Integrating non-English scientific communities to capture a truly global flow of knowledge.
  • Predictive Modeling: Moving from analyzing the past to predicting future research trends and "influence bottlenecks" using advanced graph algorithms.

Summary updated

Abstract

Bibliometric and science-of-science studies have yielded valuable insights into co-authorship and citation networks, yet most analyses rely on static datasets and limited relation types. We introduce a multi-agent AI architecture that orchestrates specialized large language model (LLM) agents (ingestion, extraction, disambiguation, integration, and analysis) to build and query a comprehensive knowledge graph. Ingestion agents unify data from diverse sources such as OpenAlex, ORCID, ROR, USPTO, and custom web scrapers. Extraction agents harness LLMs to parse unstructured text. Disambiguation agents combine rule-based heuristics with LLM reasoning to resolve ambiguous authors and institutions. Integration agents assemble and cache a provenance-rich graph. An analysis agent translates natural language questions into graph queries and interprets results. This end-to-end pipeline produces a rich graph schema spanning authors, institutions, publications, patents, grants, topics, and temporal relations. Researcher mobility and knowledge diffusion are then modeled as timed automata, where each researcher node’s institutional transitions and accumulated attributes (such as publications, collaborators, and topic expertise) enable dynamic temporal reasoning. Results show that our multi-agent, graph-based system consistently outperforms standalone LLMs and research agents on complex temporal queries, entity disambiguation accuracy, and cross-entity reasoning while maintaining competitive efficiency. These capabilities position the system as a foundation for real-time, LLM-assisted knowledge analysis platforms that can support science policy, research evaluation, and meta-scientific inquiry.
BibTeX
@inproceedings{rao2026llm,
  author = {Uttam K. Rao and Madhav Marathe},
  title = {{LLM-Enabled Scientific Knowledge Diffusion Analysis}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i47.41483},
}
Figure