TL;DR
Current machine learning approaches for automated Alzheimer’s Disease (AD) screening rely on "black-box" pattern recognition, which often exploits statistical shortcuts rather than measuring actual cognitive decline. This paper introduces Agentic Cognitive Profiling (ACP), a multi-agent LLM framework that mirrors clinical protocols by decomposing cognitive tests into atomic tasks, using deterministic function calling for error-free scoring, and generating fully auditable diagnostic reports. ACP achieves 85.3% accuracy in AD screening, proving that clinical validity and high predictive performance can coexist without compromise.
What Problem Does This Paper Address?
Most automated AI systems for detecting Alzheimer's Disease treat the problem as a standard text or audio classification task. They feed speech transcripts or audio features into Pre-trained Language Models (PLMs) like BERT to map inputs directly to a diagnostic label.
The authors argue that this approach suffers from a lack of clinical construct validity—meaning the models aren't actually measuring the underlying neurocognitive deficits (like memory loss or executive dysfunction). Instead, they often fall prey to the "Clever Hans" effect, picking up on confounding proxy features like regional dialects, acoustic shortcuts, or speech recognition errors to make predictions. This makes them brittle, prone to hallucinations, and entirely uninterpretable to clinicians who require verifiable behavioral evidence to make diagnostic decisions.
Methods
To bridge the gap between clinical logic and AI, the authors propose Agentic Cognitive Profiling (ACP), a framework that operationalizes standard clinical workflows using a multi-agent architecture.
The workflow is divided into three distinct stages:
- Administration: Standardized verbal responses and demographic data (age, education) are collected.
- Examination (The Multi-Agent Core):
- An Assigner routes different parts of the test transcript to specialized, task-specific Examiner Agents (powered by Qwen3-8B).
- Crucially, the framework decouples semantic understanding from measurement. While the LLM handles the messy nuances of natural language (disfluencies, colloquialisms), all actual scoring is delegated to Deterministic Function Calls (e.g.,
list_length(),keyword_check()). This eliminates numerical hallucinations. - A Verifier Agent cross-checks the Examiner's output against the raw transcript in a loop (up to 3 retries) to catch and correct any logic errors.
- Inference: A Meta Analyst Agent aggregates the verified scores, normalizes them against demographic tables, and passes them to a classifier (or applies clinical thresholds for a zero-shot approach). It outputs both a binary prediction and a structured, domain-by-domain cognitive profile report.
Datasets
Because no public datasets contain structured responses across multiple cognitive domains, the authors evaluated their framework on an in-house Cantonese speech corpus:
- Participants: 402 clinically annotated profiles selected from a larger cohort of 1,063 older adults.
- Tasks: Responses spanned 8 structured cognitive tasks across two gold-standard batteries: the MoCA-SL (Spoken Language subset of the Montreal Cognitive Assessment) and the HKLLT (Hong Kong List Learning Test).
- Splits: 334 participants for training (used only for the supervised baseline comparison) and 68 for testing, with balanced age and education distributions between healthy controls and AD-risk groups.
Key Insights and Ideas
- Math and Language Don't Mix in LLMs: LLMs excel at parsing colloquial Cantonese and filtering out speech disfluencies, but they struggle with reliable counting. Delegating quantification to deterministic code functions while letting the LLM handle semantics is the key to achieving clinical-grade reliability.
- Clinical Logic is a Powerful Prior: Standard machine learning assumes you need massive datasets to learn patterns from scratch. This paper shows that hardcoding clinical protocol logic into an agentic workflow allows a model to perform exceptionally well zero-shot, completely outperforming deep learning models trained on the data.
- Interpretability via Primitives: By breaking down a diagnostic score into "scoring primitives" (e.g., specific words recalled, exact subtraction errors), the framework creates an explicit audit trail.
Key Results
The ACP framework was evaluated on both its ability to score tests accurately and its ability to predict Alzheimer's.
- Scoring Accuracy: The framework achieved a 90.5% weighted-average Score Match Rate (SMR) compared to human clinical annotators, with low-inference tasks like Picture Naming and Digit Span scoring above 97%.
- AD Prediction Accuracy:
- In a supervised setting (using an SVM trained on the framework's extracted primitives), ACP achieved 85.3% accuracy, outperforming standard BERT (79.4%) and RoBERTa (75.0%) baselines.
- In a zero-shot setting (applying standard clinical cutoffs directly to the automated scores), ACP achieved 82.4% accuracy, outperforming all baseline models without using a single byte of training data.
A SHAP feature importance analysis (Figure 6) confirmed that the model's decisions were driven by clinically valid markers—primarily episodic memory recall features (n_hkllt4_recall)—rather than arbitrary statistical shortcuts.
Limitations and Future Work
- Dependence on Predefined Rules: The framework is strictly bound to existing clinical scoring guidelines. It cannot discover novel, subtle linguistic markers of cognitive decline that aren't already captured by standard tests.
- LLM Capability Ceiling: The system's accuracy is ultimately capped by the base LLM’s ability to understand complex or highly ambiguous semantic responses (e.g., abstract reasoning tasks).
- Dataset Restrictions: Due to strict privacy and IRB constraints on medical data, the Cantonese evaluation corpus cannot be publicly released, which limits direct reproducibility by outside teams.
Summary updated