Editorial: Towards Emotion AI to next generation healthcare and education

Yang Liu, Janne Kauttonen, Bowen Zhao, Xiaobai Li, Peng WeiFrontiers in Psychology 2024 · DOI · OpenAlex

Abstract

On the technical side, both traditional, shallow feature-based methods (e.g., Support Vector Machines and Random Forest) and deep neural network models (e.g., LSTM and CNN types) have been utilized. A key focus has been the development of multimodal systems that integrate diverse data sources, such as facial expressions, voice tone, dialogue sentiment, and physiological signals. Multiple signals enable the aggregation of complementary information, leveraging the strengths of various measurement techniques to enhance robustness and accuracy. Technology is only effective with users; thus, beyond developing methods, fostering adoption among professionals and citizens is essential. Alongside technical advancements, researchers have explored psychological and social factors influencing the acceptance and trust of AI systems, particularly in healthcare and education. Trust has proven pivotal for AI adoption, with studies investigating how to build it and what system attributes ensure trustworthiness. Key trust factors include explainability, transparency in processes and data usage, and the credibility of the institutions behind AI development. This Research Topic explores novel theories, methodologies, and applications of Emotion AI in healthcare and education, addressing technical aspects, models, and AI acceptance and literacy.

Introduction: The "Last Mile" of Affective Computing

Emotion AI, or Affective Computing, is transitioning from controlled laboratory environments into the complex, "in-the-wild" realities of healthcare and education. While the field has existed for decades, it currently faces a significant "last mile" problem. Technically, models that perform exceptionally on curated datasets often struggle in real-world settings due to environmental factors like poor lighting and physical occlusions. Socially, the adoption of these tools depends heavily on the trust and acceptance of clinicians, educators, and patients.

This editorial synthesizes recent research addressing these dual challenges: enhancing technical robustness through multimodal sensing and data-efficient learning, while simultaneously investigating the psychological factors—such as transparency and AI literacy—that underpin successful human-AI interaction.

Technical Advancements: Robustness and Data Efficiency

A primary focus of recent research is the development of multimodal systems that integrate facial expressions, voice tone, dialogue sentiment, and physiological signals (e.g., EEG). By aggregating complementary information, these systems aim to enhance accuracy in noisy environments.

Bridging the Performance Gap

Studies on parent-infant interactions highlight the difficulty of real-world application. Automated facial coding software currently detects only about 25% of expressions compared to manual human coding in naturalistic settings. This gap underscores the need for models capable of handling rapid movement and varied lighting conditions, ensuring that Emotion AI can function reliably outside of the lab.

Learning with Limited Data

In clinical settings, expert-labeled data is often scarce and expensive. Innovative approaches in semi-supervised and self-supervised learning are proving effective. For instance, semi-supervised models for stress monitoring have achieved 77% accuracy using only 17% labeled data, matching the performance of fully supervised approaches. Additionally, self-supervised techniques like Contrastive Predictive Coding (CPC) have shown promise in extracting meaningful features from EEG data without requiring manual labels, offering a practical path for continuous monitoring.

Human-Centric Design: Trust and Literacy

Technology is only as effective as its adoption. Research into the psychological and social factors of AI acceptance reveals that trust is a pivotal driver, particularly in sensitive fields like healthcare.

The Transparency Premium

In high-stakes environments, such as AI-powered pharmacy services, patients naturally exhibit lower trust in AI compared to human providers. However, providing informed consent and transparent communication significantly bridges this gap. Explainability is not merely a technical feature but a functional requirement for user trust; when patients understand how a system works, their willingness to use it increases.

AI Literacy and Psychological Needs

In education, fostering AI literacy is crucial for student engagement. Research indicates that satisfying students' basic psychological needs—specifically autonomy and competence—is more effective at improving AI literacy than providing direct technical support. This suggests that AI-enhanced learning environments must be designed to empower the user rather than just provide instruction.

Diverse User Attitudes

Understanding user diversity is essential for designing emotionally intelligent assistants. Research among older adults identified four distinct personas: Cautious Optimists, Skeptical Traditionalists, Positive Optimists, and Technophiles. Tailoring AI interactions to these specific attitudes can significantly improve acceptance and utility across different demographic segments.

Future Directions and Ethical Considerations

The emergence of Generative AI (GenAI) and foundation models like GPT-4 presents a new frontier for Emotion AI. These models offer the versatility to recognize emotional indicators across diverse contexts and may help mitigate the data scarcity issues currently hindering specialized model development. Furthermore, there is growing interest in the emotional analysis of groups and crowds, broadening the field's scope.

However, significant hurdles remain. Clinical applications are often limited by small sample sizes and a lack of longitudinal studies. Furthermore, the use of Emotion AI raises critical ethical concerns regarding data privacy, surveillance, and the potential misuse of sensitive personal information. Future research must prioritize cross-cultural validation, long-term real-world evaluation, and robust ethical frameworks to ensure that Emotion AI serves as a beneficial and trusted tool in next-generation healthcare and education.

Summary updated

Abstract

On the technical side, both traditional, shallow feature-based methods (e.g., Support Vector Machines and Random Forest) and deep neural network models (e.g., LSTM and CNN types) have been utilized (Khare et al., 2024;Pepa et al., 2023). A key focus has been the development of multimodal systems that integrate diverse data sources, such as facial expressions, voice tone, dialogue sentiment, and physiological signals (Geetha et al., 2024). Multiple signals enable the aggregation of complementary information, leveraging the strengths of various measurement techniques to enhance robustness and accuracy.Technology is only effective with users; thus, beyond developing methods, fostering adoption among professionals and citizens is essential. Alongside technical advancements, researchers have explored psychological and social factors influencing the acceptance and trust of AI systems, particularly in healthcare and education. Trust has proven pivotal for AI adoption, with studies investigating how to build it and what system attributes ensure trustworthiness (Li et al., 2024). Key trust factors include explainability, transparency in processes and data usage, and the credibility of the institutions behind AI development.This Research Topic explores novel theories, methodologies, and applications of Emotion AI in healthcare and education. The published works address technical aspects, models, and AI acceptance and literacy. Burgess et al. (2023) evaluated automated facial coding software for parent-infant interactions across five studies with fathers and mothers. Automated detection rates were low (~25%) compared to manual coding in naturalistic settings but strongly correlated with manual assessments, particularly for positive expressions, when successful. Key challenges included poor lighting, facial occlusion, and rapid movements, highlighting the need for greater robustness in real-world conditions. Despite these limitations, the study demonstrated the potential of automated systems for analyzing authentic emotional expressions in parent-child interactions. 2023) introduced an innovative semi-supervised learning approach for stress monitoring. Using physiological data from 14 participants across five experiments, they achieved 77% accuracy with label propagation and 76% with deep autoencoders despite utilizing only 17% labeled data. Their method matched the performance of fully supervised approaches while substantially reducing annotation requirements, offering a practical solution for continuous stress monitoring in real-world healthcare applications. Žvanut and Mihelič (2024) identified four distinct attitudes among older adults toward domestic social robots: Cautious Optimists, Skeptical Traditionalists, Positive Optimists, and Technophiles. Through interviews with 24 participants, they highlighted the influence of factors like technology familiarity, privacy concerns, and perceived utility on AI acceptance. Their findings offer valuable insights for designing emotionally intelligent robotic assistants tailored to diverse user needs and concerns. Shen and Cui (2024) investigated the link between psychological needs satisfaction and AI literacy in 445 university students. Their findings revealed that technical and teacher support positively impacted students' autonomy and competence, subsequently improving AI literacy. Notably, satisfying psychological needs proved more critical than direct support in enhancing AI literacy. These insights offer valuable guidance for designing AI-enhanced educational environments that foster better learning and engagement. Gong et al. (2024) examined patients' trust in AI-powered pharmacy intravenous admixture services across five studies. They found that patients generally trusted AI PIVAS less than human services, primarily due to a limited subjective understanding of AI systems. However, informed consent significantly improved trust by enhancing patients' understanding. This study underscores the critical role of transparent communication and psychological factors in adopting AI in healthcare.The articles in this Research Topic advance methods for analyzing real-world datasets and explore how Emotion AI can be effectively integrated into healthcare and education, addressing challenges related to trust, privacy, and user acceptance. While Emotion AI shows promise, its use in these fields raises technical and ethical concerns, including data privacy, surveillance, and potential misuse of personal information. Clinical applications are further constrained by small sample sizes, lack of control groups, limited real-world testing, and methodological variability, which hinder reproducibility and reliability (Pepa et al., 2023). These studies tackle these challenges by developing novel model-training approaches for limited and noisy labeled data and identifying factors influencing AI acceptance and trust.Future research should prioritize enhancing the robustness of emotion recognition in naturalistic settings, developing efficient learning methods to reduce dependence on labeled data, and addressing ethical considerations to support user acceptance of AI. Emphasis on cross-cultural validation and long-term evaluation in real-world applications is crucial. Recent generative AI (GenAI) foundation models, such as GPT-4, offer significant potential for advancing Emotion AI due to their versatility (Cheng et al., 2023). GenAI has already shown promise in recognizing emotional indicators (Elyoseph et al., 2024) and could help alleviate data scarcity challenges that hinder specialized model development. Additionally, there is growing interest in the emotional analysis of groups and crowds, broadening the field's scope (Veltmeijer et al., 2023;Li et al. 2024).
BibTeX
@inproceedings{liu2024editorial,
  author = {Yang Liu and Janne Kauttonen and Bowen Zhao and Xiaobai Li and Peng Wei},
  title = {{Editorial: Towards Emotion AI to next generation healthcare and education}},
  booktitle = {Frontiers in Psychology},
  year = {2024},
  doi = {10.3389/fpsyg.2024.1533053},
}
Figure