TL;DR
This paper introduces a novel, hierarchical, and multimodal dialogue act taxonomy specifically designed for a proactive virtual coach aimed at improving the quality of life for elderly individuals. By adapting the GROW coaching model, this taxonomy enables the agent to actively guide conversations towards specific coaching goals, moving beyond traditional task-oriented or chit-chat systems.
What Problem Does This Paper Address?
The development of intelligent virtual assistants for the elderly faces unique challenges. Beyond basic task completion, there's a need to incorporate emotional intelligence and, crucially, to empower the agent to proactively guide users towards achieving mid- and long-term goals. This requires a sophisticated dialogue management strategy that goes beyond simply responding to user intents. Specifically, this paper addresses the need for a dialogue act taxonomy that can capture the nuances of a coaching-based interaction, where the agent takes the lead in driving the conversation towards predefined coaching objectives, particularly for an elderly demographic.
Methods
The core contribution is a new dialogue act taxonomy. This taxonomy is characterized by:
- Coaching-Centric Design: It's built around the GROW (Goals, Reality, Options, Will/Way forward) coaching model, enabling the agent to structure conversations around goal setting and achievement.
- Hierarchical Structure: Dialogue acts are organized hierarchically, allowing for both general and specific semantic information to be captured. This provides flexibility, enabling the system to infer meaning even with less precise input.
- Multimodality: The taxonomy considers multiple modes of semantic information, including Topic, Intent, Polarity, and Entity labels. This richer representation is crucial for understanding the user's state and guiding the conversation effectively.
- Proactive Agent Role: Unlike systems where the user drives the conversation, this taxonomy is designed for an agent that actively leads the dialogue to achieve coaching goals.
The taxonomy was applied to annotate a corpus of Spanish dialogues collected from elderly participants interacting with a simulated virtual coach (Wizard of Oz setup).
Datasets
The paper utilizes a newly annotated Spanish dialogue corpus collected from 192 elderly participants. This corpus consists of approximately 384 dialogues, each around 10 minutes long, totaling about 4522 turns. The dialogues were recorded in two scenarios: an introductory session to build comfort and a coaching session focused on nutrition, following the GROW model. This corpus is unique in its focus on elderly users and its specific application to a coaching context.
Key Insights and Ideas
- Coaching Requires Proactivity: The paper highlights that effective coaching by a virtual agent necessitates a proactive stance, actively guiding the conversation rather than passively responding.
- Hierarchical Labels Offer Flexibility: The hierarchical nature of the taxonomy allows for more robust dialogue management, especially when dealing with ambiguous user input. The system can fall back on more general labels if specific ones cannot be confidently determined.
- Multimodality is Key for Elderly Users: Capturing Topic, Intent, Polarity, and Entity information provides a richer understanding of the user's state and needs, which is particularly important for an elderly demographic where subtle cues might be crucial.
- GROW Model Translates to Dialogue Acts: The paper demonstrates that the structured phases of the GROW model can be effectively mapped to specific dialogue act categories, facilitating goal-oriented conversations.
- Elderly Conversations are Simpler: The analysis of the annotated corpus revealed a tendency for shorter turns and subsentences, consistent with typical human-machine interactions and potentially reflecting the communication style of the target demographic.
Key Results
- The annotated corpus shows a significant prevalence of "Other" as a topic label (63.7%), often associated with general conversation and greetings.
- Among intent labels, "generic - agreement" (17.4%) and "generic - evaluation/opinion" (18.1%) were frequent, alongside "GROW inform - habit or action" (16.7%), indicating user engagement with the coaching aspects.
- Polarity analysis showed users were predominantly neutral (around 60%), with positive sentiment being more common than negative.
- An average of 2.6 entities were identified per user turn, with "Nourishment" and "Actions" being prominent, aligning with the nutrition coaching scenario.
- Sankey diagrams illustrated strong relationships between specific intents and topics (e.g., "GROW inform - habit or action" with "Nutrition") and between intents and entities (e.g., "GROW inform" with "Nourishment" and "Actions").
Limitations and Future Work
- Corpus Size: While valuable, the Spanish corpus is relatively small (4522 turns) for training complex machine learning models.
- Annotation Complexity: Hierarchical, multimodal taxonomies, while powerful, present challenges for typical machine learning methods, requiring more sophisticated classification strategies.
- Language and Cultural Differences: The current analysis is primarily based on Spanish dialogues. Future work includes completing annotations for other languages (French and Norwegian) to enable cross-cultural comparisons.
- Automatic Labeling: The current annotation relies on human annotators. Future work will involve developing classification strategies to automatically label dialogues using the proposed taxonomy.
- Transfer Learning: The availability of multilingual data opens avenues for investigating transfer learning strategies for dialogue classifiers.
- Deeper Analysis: Further analysis is needed to correlate dialogue act labels with emotion annotations derived from speech and facial expressions.
Summary updated