Abstract
Our system illustrates a powerful paradigm in which AI can facilitate automated data analysis, expediting advancements in population-level dementia investigations.
Introduction
Scientific discovery in Alzheimer’s Disease and Related Dementias (ADRD) is notoriously slow, often hindered by the complexity of large-scale clinical datasets. While repositories like the National Alzheimer’s Coordinating Center (NACC) provide vast amounts of multimodal data, they are inherently "noisy." Researchers typically spend months on data harmonization, variable selection, and preprocessing before a single hypothesis can be tested.
This "analytic friction" creates a significant bottleneck, favoring well-resourced teams and delaying the discovery of complex disease pathways involving genetics, lifestyle, and vascular health. To address this, we developed a multi-agent AI system that acts as a collaborative team of scientists to automate the end-to-end research workflow, from data preparation to statistical analysis and peer-style critique.
Methods: A Multi-Agentic Architecture
The system is powered by the Qwen3-4B-Instruct large language model and utilizes a specialized multi-agent architecture to divide labor across four distinct roles:
- Data Preparation Agent: Automatically detects CSV files and JSON dictionaries, selects relevant features for a given hypothesis, and handles preprocessing tasks such as one-hot encoding and variable standardization.
- Planning Agent: Designs the research strategy and selects appropriate statistical tests (e.g., Chi-square, regression) based on the underlying data types.
- Scientist Agent: Writes and executes Python code in a secure environment to perform the actual statistical analysis.
- Critic Agent: Acts as an internal peer reviewer, evaluating the code for errors, checking if statistical assumptions were met, and ensuring the results align with the original hypothesis.
The system operates through nested loops: an "inner loop" where the Scientist and Critic iterate on code and analysis, and an "outer loop" where the Planning agent can pivot to different variables if the initial approach proves insufficient.
Experimental Design
The study utilized the National Alzheimer’s Coordinating Center (NACC) cohort, a robust dataset containing records for 48,876 participants and over 750 variables ranging from demographics to neuropsychological tests.
To evaluate the system's performance, we curated 100 hypotheses from established ADRD literature (2000–2024). Additionally, we introduced 100 non-ADRD control hypotheses (e.g., related to oncology or cardiology) to test the system's specificity and its ability to avoid false-positive discoveries.
Key Results and Insights
Hypothesis Replication and Consistency
On the full NACC cohort, the system verified 42.0 ± 2.8 of the 100 literature-derived hypotheses by rejecting the null hypothesis with statistical significance. The system also demonstrated high logical consistency when testing "negated" versions of hypotheses (e.g., testing if a risk factor decreases risk instead of increasing it).
The "Not Testable" Safety Valve
A critical feature of the system is its ability to declare a hypothesis "NOT TESTABLE" rather than hallucinating a result. In the negative control test using non-ADRD hypotheses, the system flagged 85.2% as not testable. This high specificity suggests the AI is less likely to produce spurious findings when faced with irrelevant queries.
Statistical Power and Scaling
The system’s performance scaled logically with dataset size. At a sample size of $n=100$, it verified only 10.4% of hypotheses due to low statistical power. However, at $n=10,000$, the verification rate rose to 35.4%, mimicking the behavior of a human researcher who gains confidence as sample sizes grow.
Domain-Awareness
Unlike general-purpose coding assistants, this system utilizes ADRD-specific ontologies to map natural language queries (e.g., "alcohol consumption") to specific database codes (e.g., ALCFREQ), ensuring that the analysis remains grounded in clinical reality.
Limitations and Future Work
While the system successfully replicates known findings, it is currently limited by literature bias, as its success is measured against existing discoveries rather than the generation of entirely novel insights. Furthermore, the system is bound by the inherent biases of the NACC dataset, which may underrepresent certain demographics.
Future iterations will focus on integrating federated learning to allow agents to analyze data across multiple sites without compromising patient privacy. Additionally, we aim to adapt the agents to handle real-time data from wearables, further expanding the scope of automated dementia research.
Summary updated