TL;DR
This study investigates which specific relational cues in e-coach dialogues are perceived as helpful by users and how these cues influence the working alliance and intention to change health behaviors. The research found that while the overall impact of empathic vs. neutral e-coaches wasn't significant, certain relational cues, particularly those related to working alliance and social dialogue, were more impactful for specific user groups and behaviors.
What Problem Does This Paper Address?
Embodied Conversational Agents (ECAs), or virtual coaches, are increasingly used in digital health to motivate users and encourage adherence to treatment plans. These ECAs often incorporate "relational cues" – elements like empathy, respect, and openness – to build a stronger bond and improve outcomes. However, there's a significant gap in understanding which specific relational cues are actually helpful to whom, and how these cues impact the user's experience and behavior change intentions. Without this understanding, designing effective and personalized e-coaches is challenging, potentially leading to a "one-size-fits-all" approach that may not be optimal. This paper aims to address this by dissecting the helpfulness of different relational cues and exploring their impact on user engagement and behavior change.
Methods
The study employed a pre-post experimental design using a custom-built e-coach named Alex. Alex was implemented with two versions: an "empathic Alex" that included a comprehensive set of relational cues, and a "neutral Alex" that omitted these cues. Participants were randomly assigned to interact with one of the versions.
The relational cues were categorized into four main types:
- Empowerment: Encouraging users to make decisions or choose options.
- Working Alliance: Building mutual understanding, shared goals, and trust.
- Affirmation: Acknowledging user input, expressing empathy, and offering encouragement.
- Social Dialogue: Non-task-oriented conversation for openness, including greetings, self-disclosure, and politeness.
The study collected data on user demographics, personality traits (using the Ten-Item Personality Inventory - TIPI), health behavior intention (pre and post-interaction), working alliance (using the Session Rating Scale - SRS), and the perceived helpfulness of specific relational cues. Machine learning models (binary logistic regression and decision trees) were used to analyze the relationships between individual factors, cue helpfulness, and outcomes like behavior change intention and working alliance.
Datasets
The study utilized data collected from 206 participants who interacted with the e-coach Alex. This dataset included:
- Demographic information (gender, age, cultural background).
- Personality assessments (TIPI).
- Health behavior questionnaires (pre and post-intervention).
- Working alliance ratings (SRS).
- Perceived helpfulness ratings for 16 specific relational cues.
No new benchmark datasets were introduced; the study relied on data collected from its experimental setup.
Key Insights and Ideas
- Nuance in Relational Cues: While the overall difference between empathic and neutral e-coaches wasn't statistically significant for behavior change or working alliance, the analysis of individual cues revealed important distinctions. This suggests that the specific cues matter more than simply having an "empathic" agent.
- Working Alliance and Social Dialogue Shine: Cues related to building a working alliance (e.g., "I hope together we can make progress") and social dialogue (e.g., self-disclosure from the coach) showed significant differences in helpfulness between groups, with some being rated higher by users of the neutral agent. This hints at a complex interplay where the delivery and context of a cue can influence its perceived helpfulness.
- Empowerment is Universally Liked: The empowerment cue "If you set your own goal, chances are higher that you actually stick with them" was the most liked cue across both empathic and neutral groups, highlighting its strong motivational impact.
- Individual Factors Matter: The study found that individual factors like gender, age, and personality traits significantly influence which relational cues are perceived as helpful and their impact on behavior change and working alliance. For instance, females showed a greater intention to change for certain behaviors, and introverts were more cautious in decision-making.
- Self-Disclosure Requires Care: Cues involving self-disclosure, while potentially beneficial for building rapport, need careful design. Some users found them less helpful or even uncomfortable when delivered by an ECA, suggesting a need for ethical considerations and user preference alignment.
Key Results
- No Overall Difference in Outcomes: No significant differences were found between the empathic and neutral Alex groups in terms of overall behavior change intention or working alliance scores.
- Usability Differences: The neutral Alex group reported significantly less system complexity, easier use, and greater confidence, suggesting that the absence of certain relational cues might, in some aspects, lead to a smoother user experience.
- Specific Cue Differences: Several specific relational cues showed statistically significant differences in helpfulness ratings between the empathic and neutral groups. Notably, some working alliance and social dialogue cues were rated higher by the neutral group.
- Most Liked Cue: The empowerment cue "If you set your own goal, chances are higher that you actually stick with them" was rated as helpful or very helpful by 82% of users across both groups.
- Predictive Models: Machine learning models indicated that relational cues, particularly working alliance and empowerment, were significant predictors of behavior change intention and working alliance, with varying contributions from individual factors like gender and personality.
Limitations and Future Work
- Sample Limitations: The study's sample consisted primarily of university students, limiting the generalizability of findings to other demographics, particularly older adults or those with different health conditions. Future work should involve diverse populations and potentially adapt target behaviors.
- Limited Cue Set and Dialogue Duration: The study evaluated a specific set of relational cues and had a limited dialogue duration. Future research could explore a wider range of cues and longer interaction periods.
- Generalizability of Models: The models developed are specific to the behaviors and cues studied. Further research is needed to uncover models specific to different target users and contexts.
- Personalization and Context-Awareness: The authors suggest future work should focus on automatic generation of relational cues based on user preferences and context. This includes incorporating more personality aspects into dialogues and adapting social dialogues based on individual factors to enhance likability and engagement.
- Multi-Domain ECAs: Extending the work to include multiple coaches with expertise in different health domains (e.g., diet, physiotherapy) is proposed to provide integrated health support.
Summary updated