Towards a Psychological Generalist AI: A Survey of Current Applications of Large Language Models and Future Prospects

Tianyu He, Guanghui Fu, Yijing Yu, Fan Wang, Jianqiang Li, Qing Zhao, Changwei Song, Hongzhi Qi, Dan Luo, Huijing Zou, Bing Xiang YangarXiv 2023 · DOI · arXiv

Abstract

This paper surveys the evolution of "Generalist Psychology Artificial Intelligence" (GPAI), moving beyond narrow, task-specific models toward versatile Large Language Models (LLMs) that can reason, empathize, and intervene across diverse psychological scenarios. It highlights how LLMs like GPT-4 are beginning to match human professionals in specific diagnostic tasks while outlining the ethical and technical roadmaps needed to make these tools safe for clinical use.

Introduction

Mental health care faces a global crisis: a massive shortage of professional counselors, high costs, and the complexity of human psychology that traditional "narrow" AI struggles to capture. Historically, AI in psychology relied on supervised learning with expert-labeled datasets for single tasks, such as detecting depression in social media posts. These specialized models are often too rigid for the multi-modal and context-dependent nature of human psychology.

Overview of GPAI applications across clinical monitoring, diagnostic consultation, and mobile supportive settings.

This paper explores the transition to GPAI—models that don't just perform one task but can adapt to different age groups, settings (hospitals vs. social media), and stages of disease (prevention vs. relapse) using the broad reasoning capabilities of LLMs.

Methodology and Framework

The proposed framework for GPAI is built upon three core pillars:

  1. Ethical Monitoring: Tracking mental health across platforms while prioritizing privacy.
  2. Multi-modal Collaboration: Integrating text, audio from interviews, visual data from social media, and time-series data from wearables.
  3. Advanced Reasoning: Using LLMs to deduce novel mental states and predict disease trajectories, such as forecasting suicide risk based on hotline interactions.
Integration of multi-modal data and psychological metrics in a generalist AI dashboard.

Researchers utilize several benchmarks to evaluate these models, including the CLEVA platform for general competence, CHBias for detecting model bias, and the Level of Emotional Awareness Scale (LEAS) to benchmark emotional intelligence against human norms.

Key Results and Insights

LLMs have demonstrated significant potential across various psychological tasks:

  • Emotional Awareness: GPT-3.5 reached a LEAS Z-score of 4.26, showing an "exceptional accuracy level" of 9.7/10 when evaluated by licensed psychologists.
  • Diagnostic Accuracy: In classification tasks, LLMs achieved F1 scores of 0.73 for stress detection and 0.86 for depression detection.
  • Suicide Risk Assessment: GPT-4’s assessments were found to be in "close agreement" with human professionals, significantly outperforming earlier models.
  • Theory of Mind (ToM): While LLMs show some ability to "mentalize" (understand others' beliefs), their accuracy in complex scenarios still lags behind human levels.

Beyond diagnosis, LLMs serve as virtual companions (e.g., Replika) and tools for social simulacra, where researchers populate virtual environments with AI agents to test psychological theories like "homophily" without the need for expensive human trials.

Limitations and Future Directions

Despite these advancements, several hurdles remain. GPT-4 has a tendency to overestimate "psychache" (psychological pain), which could lead to false alarms in clinical settings. Furthermore, the "insatiable thirst for data" in LLMs conflicts with the necessity of patient confidentiality. The authors suggest Federated Learning as a potential solution to train models without compromising sensitive data.

Future research must address algorithmic bias and focus on multi-domain evaluation across different cultures and languages. The development of open-source pre-trained models is also crucial to ensure that GPAI technology remains accessible and is not controlled solely by a few large corporations.

Conclusion

The shift toward Generalist Psychology AI represents a transformative step in mental health technology. By leveraging the reasoning and zero-shot capabilities of LLMs, GPAI offers a path toward scalable, empathetic, and effective psychological support, provided that ethical and technical challenges are rigorously addressed.

Summary updated

BibTeX
@misc{he2023towards,
  author = {Tianyu He and Guanghui Fu and Yijing Yu and Fan Wang and Jianqiang Li and Qing Zhao and Changwei Song and Hongzhi Qi and Dan Luo and Huijing Zou and Bing Xiang Yang},
  title = {{Towards a Psychological Generalist AI: A Survey of Current Applications of Large Language Models and Future Prospects}},
  year = {2023},
  eprint = {2312.04578},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.04578},
}
Figure