Identifying Which Relational Cues Users Find Helpful to Allow Tailoring of e-Coach Dialogues

Sana Salman, Debbie Richards, Mark Dras

TL;DR

This study investigates which specific relational cues in e-coach dialogues are perceived as helpful by users and how these cues influence the working alliance and intention to change health behaviors. The research found that while the overall impact of empathic vs. neutral e-coaches wasn't significant, certain relational cues, particularly those related to working alliance and social dialogue, were more impactful for specific user groups and behaviors.

What Problem Does This Paper Address?

Embodied Conversational Agents (ECAs), or virtual coaches, are increasingly used in digital health to motivate users and encourage adherence to treatment plans. These ECAs often incorporate "relational cues" – elements like empathy, respect, and openness – to build a stronger bond and improve outcomes. However, there's a significant gap in understanding which specific relational cues are actually helpful to whom, and how these cues impact the user's experience and behavior change intentions. Without this understanding, designing effective and personalized e-coaches is challenging, potentially leading to a "one-size-fits-all" approach that may not be optimal. This paper aims to address this by dissecting the helpfulness of different relational cues and exploring their impact on user engagement and behavior change.

Methods

The study employed a pre-post experimental design using a custom-built e-coach named Alex. Alex was implemented with two versions: an "empathic Alex" that included a comprehensive set of relational cues, and a "neutral Alex" that omitted these cues. Participants were randomly assigned to interact with one of the versions.

The relational cues were categorized into four main types:

  • Empowerment: Encouraging users to make decisions or choose options.
  • Working Alliance: Building mutual understanding, shared goals, and trust.
  • Affirmation: Acknowledging user input, expressing empathy, and offering encouragement.
  • Social Dialogue: Non-task-oriented conversation for openness, including greetings, self-disclosure, and politeness.

The study collected data on user demographics, personality traits (using the Ten-Item Personality Inventory - TIPI), health behavior intention (pre and post-interaction), working alliance (using the Session Rating Scale - SRS), and the perceived helpfulness of specific relational cues. Machine learning models (binary logistic regression and decision trees) were used to analyze the relationships between individual factors, cue helpfulness, and outcomes like behavior change intention and working alliance.

Datasets

The study utilized data collected from 206 participants who interacted with the e-coach Alex. This dataset included:

  • Demographic information (gender, age, cultural background).
  • Personality assessments (TIPI).
  • Health behavior questionnaires (pre and post-intervention).
  • Working alliance ratings (SRS).
  • Perceived helpfulness ratings for 16 specific relational cues.

No new benchmark datasets were introduced; the study relied on data collected from its experimental setup.

Key Insights and Ideas

  • Nuance in Relational Cues: While the overall difference between empathic and neutral e-coaches wasn't statistically significant for behavior change or working alliance, the analysis of individual cues revealed important distinctions. This suggests that the specific cues matter more than simply having an "empathic" agent.
  • Working Alliance and Social Dialogue Shine: Cues related to building a working alliance (e.g., "I hope together we can make progress") and social dialogue (e.g., self-disclosure from the coach) showed significant differences in helpfulness between groups, with some being rated higher by users of the neutral agent. This hints at a complex interplay where the delivery and context of a cue can influence its perceived helpfulness.
  • Empowerment is Universally Liked: The empowerment cue "If you set your own goal, chances are higher that you actually stick with them" was the most liked cue across both empathic and neutral groups, highlighting its strong motivational impact.
  • Individual Factors Matter: The study found that individual factors like gender, age, and personality traits significantly influence which relational cues are perceived as helpful and their impact on behavior change and working alliance. For instance, females showed a greater intention to change for certain behaviors, and introverts were more cautious in decision-making.
  • Self-Disclosure Requires Care: Cues involving self-disclosure, while potentially beneficial for building rapport, need careful design. Some users found them less helpful or even uncomfortable when delivered by an ECA, suggesting a need for ethical considerations and user preference alignment.

Key Results

  • No Overall Difference in Outcomes: No significant differences were found between the empathic and neutral Alex groups in terms of overall behavior change intention or working alliance scores.
  • Usability Differences: The neutral Alex group reported significantly less system complexity, easier use, and greater confidence, suggesting that the absence of certain relational cues might, in some aspects, lead to a smoother user experience.
  • Specific Cue Differences: Several specific relational cues showed statistically significant differences in helpfulness ratings between the empathic and neutral groups. Notably, some working alliance and social dialogue cues were rated higher by the neutral group.
  • Most Liked Cue: The empowerment cue "If you set your own goal, chances are higher that you actually stick with them" was rated as helpful or very helpful by 82% of users across both groups.
  • Predictive Models: Machine learning models indicated that relational cues, particularly working alliance and empowerment, were significant predictors of behavior change intention and working alliance, with varying contributions from individual factors like gender and personality.

Limitations and Future Work

  • Sample Limitations: The study's sample consisted primarily of university students, limiting the generalizability of findings to other demographics, particularly older adults or those with different health conditions. Future work should involve diverse populations and potentially adapt target behaviors.
  • Limited Cue Set and Dialogue Duration: The study evaluated a specific set of relational cues and had a limited dialogue duration. Future research could explore a wider range of cues and longer interaction periods.
  • Generalizability of Models: The models developed are specific to the behaviors and cues studied. Further research is needed to uncover models specific to different target users and contexts.
  • Personalization and Context-Awareness: The authors suggest future work should focus on automatic generation of relational cues based on user preferences and context. This includes incorporating more personality aspects into dialogues and adapting social dialogues based on individual factors to enhance likability and engagement.
  • Multi-Domain ECAs: Extending the work to include multiple coaches with expertise in different health domains (e.g., diet, physiotherapy) is proposed to provide integrated health support.

Summary updated

Abstract

Relational cues are extracts from actual verbal dialogues that help build the therapist–patient working alliance and stronger bond through the depiction of empathy, respect and openness. ECAs (Embodied conversational agents) are human-like virtual agents that exhibit verbal and non-verbal behaviours. In the digital health space, ECAs act as health coaches or experts. ECA dialogues have previously been designed to include relational cues to motivate patients to change their current behaviours and encourage adherence to a treatment plan. However, there is little understanding of who finds specific relational cues delivered by an ECA helpful or not. Drawing the literature together, we have categorised relational cues into empowering, working alliance, affirmative and social dialogue. In this study, we have embedded the dialogue of Alex, an ECA, to encourage healthy behaviours with all the relational cues (empathic Alex) or with none of the relational cues (neutral Alex). A total of 206 participants were randomly assigned to interact with either empathic or neutral Alex and were also asked to rate the helpfulness of selected relational cues. We explore if the perceived helpfulness of the relational cues is a good predictor of users’ intention to change the recommended health behaviours and/or development of a working alliance. Our models also investigate the impact of individual factors, including gender, age, culture and personality traits of the users. The idea is to establish whether a certain group of individuals having similarities in terms of individual factors found a particular cue or group of cues helpful. This will establish future versions of Alex and allow Alex to tailor its dialogue to specific groups, as well as help in building ECAs with multiple personalities and roles.
BibTeX
@article{salman2023identifying,
  author = {Sana Salman and Debbie Richards and Mark Dras},
  title = {{Identifying Which Relational Cues Users Find Helpful to Allow Tailoring of e-Coach Dialogues}},
  journal = {Multimodal Technol. Interact},
  year = {2023},
  doi = {10.3390/mti7100093},
}

Loading related content...

Figure