PENSIEVE-AI a brief cognitive test to detect cognitive impairment across diverse literacy

Tau Ming Liew, Jong Yong Abdiel Foo, Howard H. Yang, Sze Yan Tay, Way Inn Koay, King Fan Yip, Simon Kang Seng Ting, Kaavya Narasimhalu, Weishan Li, Congyuan Tan, Dan Luo, Rebecca Chong, Rachel Shong, Calvin Sia, Gerald Choon‐Huat Koh, Julian ThumbooNature Communications 2025 · DOI

TL;DR

PENSIEVE-AI is a new, self-administered digital screening tool that uses deep learning to detect mild cognitive impairment (MCI) and dementia in under five minutes. By focusing on drawing-based tasks rather than language-heavy questions, the tool maintains high diagnostic accuracy (AUC = 93%) across diverse literacy levels, matching the performance of much longer, professional-led neuropsychological assessments.

What Problem Does This Paper Address?

Cognitive impairment (CI) is a massive global health challenge, yet 60–90% of cases go undiagnosed. Early detection is critical for intervention, but current screening methods face two major hurdles:

  1. Scalability: Most "brief" tests still require trained healthcare professionals to administer and score them, making them difficult to deploy in community settings or primary care.
  2. Literacy Bias: Standard tests (like the Mini-Mental State Examination) often rely on reading, writing, and language skills. This leads to "false positives" in individuals with low formal education, particularly in underserved communities and lower-middle-income countries (LMICs).

The authors sought to create a tool that is both "literacy-neutral" and capable of being self-administered on a tablet without professional supervision.

Methods

The researchers developed PENSIEVE-AI, a digital test consisting of four drawing tasks:

  • Complex figure copy
  • Simple figure copy
  • Clock drawing
  • Complex figure recall (memory-based)

The AI Architecture: The team experimented with several models, ultimately finding that a VGG-16 convolutional neural network performed best. Crucially, they didn't just feed the model the final static images. They incorporated "Drawing Activities"—process-based data such as the time taken between strokes, pauses, and repeated attempts.

The final model integrated three components: the image features (VGG-16), the process data (Drawing Activities), and a single demographic covariate (Years of Education).

Datasets

The study utilized a prospective cohort of 1,758 community-dwelling individuals aged 65 and older in Singapore.

  • Diversity: The cohort was highly representative of a multiethnic Asian population (Chinese, Malay, Indian).
  • Education Range: Participants had a wide range of formal education, from 0 to 23 years (median = 10 years).
  • Gold Standard Comparison: All participants underwent a "gold standard" clinical adjudication process involving a 1-2 hour Neuropsychological Test Battery (NTB) and a consensus diagnosis by three specialists.
  • Splits: The data was split into Training (~40%), Validation (~20%), and a single hold-out Test set (~40%, n=658).

Key Insights and Ideas

  • Process is as Important as the Product: One of the most significant findings was that adding "drawing activities" (the temporal data of how a person draws) significantly boosted the model's AUC from 88.2% to 93.2%. This captures subtle behavioral nuances—like hesitation or confusion—that a static pen-and-paper test misses.
  • Memory Tasks are Essential: In ablation studies, the "Complex figure recall" task was the single most powerful predictor (AUC = 89.8%). This confirms that memory decline remains the hallmark of early-stage CI, even in non-verbal formats.
  • Closing the Literacy Gap: Unlike traditional tools like the iAD8, which saw a significant drop in performance for participants with ≤10 years of education, PENSIEVE-AI remained robust. Its performance was consistent across both low- and high-education subgroups.
  • The "Two-Cutoff" Strategy: Instead of a single "pass/fail" score, the authors propose a two-threshold approach:
  • Lower Cutoff: High sensitivity to "rule out" impairment.
  • Upper Cutoff: High specificity to "rule in" impairment for direct referral to memory clinics.
  • Intermediate Zone: Identifies a "grey area" group requiring closer annual monitoring.

Key Results

  • Diagnostic Accuracy: PENSIEVE-AI achieved an AUC of 93.2% for detecting MCI/dementia.
  • Benchmarking: This performance was statistically comparable to the 1-2 hour professional Neuropsychological Test Battery (AUC = 94%) and the MoCA (AUC = 95%).
  • Efficiency: 77% of participants completed the test in under 5 minutes, with a median completion time of 3.7 minutes.
  • Self-Administration: Nearly 70% of the total sample (and 73% of the cognitively normal group) successfully completed the test with no supervision at all.

Limitations and Future Work

While promising, the authors note several limitations:

  • Physical Constraints: The test is unsuitable for those with severe visual impairment or motor tremors (e.g., advanced Parkinson’s) that prevent stylus use.
  • Domain Specificity: Because it is drawing-based, it may be less sensitive to "language-first" cognitive decline, which is more common in certain types of young-onset dementia.
  • The Digital Divide: While designed for low literacy, the "technological literacy" of using a tablet still presents a barrier for some elderly users in LMICs.
  • Future Directions: The team plans to validate the tool in other cultures and languages (translating the audio instructions) and evaluate its ability to track cognitive decline longitudinally over several years.

Summary updated

Abstract

Undiagnosed cognitive impairment is a pervasive global issue, often due to subtle nature of early symptoms, necessitating the use of brief cognitive tests for early detection. However, most brief tests are not scalable (requiring trained professionals), and are not designed for lower literacy groups (e.g. in underserved communities). Here, we developed PENSIEVE-AI<sup>TM</sup>, a drawing-based digital test that is less dependent on literacy, and can be self-administered in <5 min. In a prospective study involving 1758 community-dwelling individuals aged 65 and older from Singapore (education range = 0-23 years), our deep-learning model showed excellent performance in detecting clinically-adjudicated mild cognitive impairment and dementia (AUC = 93%), comparable to traditional neuropsychological assessments (AUC = 94%, P<sub>comparison</sub> = 1.000). Results were consistent even across education subgroups. Being less dependent on literacy, PENSIEVE-AI holds promise for broader deployment in literacy-diverse populations similar to Singapore (e.g. some Asian and lower- and middle-income countries), potentially improving early detection and intervention of cognitive impairment.
BibTeX
@inproceedings{liew2025pensieve,
  author = {Tau Ming Liew and Jong Yong Abdiel Foo and Howard H. Yang and Sze Yan Tay and Way Inn Koay and King Fan Yip and Simon Kang Seng Ting and Kaavya Narasimhalu and Weishan Li and Congyuan Tan and Dan Luo and Rebecca Chong and Rachel Shong and Calvin Sia and Gerald Choon‐Huat Koh and Julian Thumboo},
  title = {{PENSIEVE-AI a brief cognitive test to detect cognitive impairment across diverse literacy}},
  booktitle = {Nature Communications},
  year = {2025},
  doi = {10.1038/s41467-025-58201-x},
}
Figure