A Dialogue-Act Taxonomy for a Virtual Coach Designed to Improve the Life of Elderly

César Montenegro, Asier López-Zorrilla, Javier Mikel Olaso, Roberto Santana, Raquel Justo, José Antonio Lozano, María Inés TorresMultimodal Technol. Interact 2019 · OpenReview

TL;DR

This paper introduces a novel, hierarchical, and multimodal dialogue act taxonomy specifically designed for a proactive virtual coach aimed at improving the quality of life for elderly individuals. By adapting the GROW coaching model, this taxonomy enables the agent to actively guide conversations towards specific coaching goals, moving beyond traditional task-oriented or chit-chat systems.

What Problem Does This Paper Address?

The development of intelligent virtual assistants for the elderly faces unique challenges. Beyond basic task completion, there's a need to incorporate emotional intelligence and, crucially, to empower the agent to proactively guide users towards achieving mid- and long-term goals. This requires a sophisticated dialogue management strategy that goes beyond simply responding to user intents. Specifically, this paper addresses the need for a dialogue act taxonomy that can capture the nuances of a coaching-based interaction, where the agent takes the lead in driving the conversation towards predefined coaching objectives, particularly for an elderly demographic.

Methods

The core contribution is a new dialogue act taxonomy. This taxonomy is characterized by:

  • Coaching-Centric Design: It's built around the GROW (Goals, Reality, Options, Will/Way forward) coaching model, enabling the agent to structure conversations around goal setting and achievement.
  • Hierarchical Structure: Dialogue acts are organized hierarchically, allowing for both general and specific semantic information to be captured. This provides flexibility, enabling the system to infer meaning even with less precise input.
  • Multimodality: The taxonomy considers multiple modes of semantic information, including Topic, Intent, Polarity, and Entity labels. This richer representation is crucial for understanding the user's state and guiding the conversation effectively.
  • Proactive Agent Role: Unlike systems where the user drives the conversation, this taxonomy is designed for an agent that actively leads the dialogue to achieve coaching goals.

The taxonomy was applied to annotate a corpus of Spanish dialogues collected from elderly participants interacting with a simulated virtual coach (Wizard of Oz setup).

Datasets

The paper utilizes a newly annotated Spanish dialogue corpus collected from 192 elderly participants. This corpus consists of approximately 384 dialogues, each around 10 minutes long, totaling about 4522 turns. The dialogues were recorded in two scenarios: an introductory session to build comfort and a coaching session focused on nutrition, following the GROW model. This corpus is unique in its focus on elderly users and its specific application to a coaching context.

Key Insights and Ideas

  • Coaching Requires Proactivity: The paper highlights that effective coaching by a virtual agent necessitates a proactive stance, actively guiding the conversation rather than passively responding.
  • Hierarchical Labels Offer Flexibility: The hierarchical nature of the taxonomy allows for more robust dialogue management, especially when dealing with ambiguous user input. The system can fall back on more general labels if specific ones cannot be confidently determined.
  • Multimodality is Key for Elderly Users: Capturing Topic, Intent, Polarity, and Entity information provides a richer understanding of the user's state and needs, which is particularly important for an elderly demographic where subtle cues might be crucial.
  • GROW Model Translates to Dialogue Acts: The paper demonstrates that the structured phases of the GROW model can be effectively mapped to specific dialogue act categories, facilitating goal-oriented conversations.
  • Elderly Conversations are Simpler: The analysis of the annotated corpus revealed a tendency for shorter turns and subsentences, consistent with typical human-machine interactions and potentially reflecting the communication style of the target demographic.

Key Results

  • The annotated corpus shows a significant prevalence of "Other" as a topic label (63.7%), often associated with general conversation and greetings.
  • Among intent labels, "generic - agreement" (17.4%) and "generic - evaluation/opinion" (18.1%) were frequent, alongside "GROW inform - habit or action" (16.7%), indicating user engagement with the coaching aspects.
  • Polarity analysis showed users were predominantly neutral (around 60%), with positive sentiment being more common than negative.
  • An average of 2.6 entities were identified per user turn, with "Nourishment" and "Actions" being prominent, aligning with the nutrition coaching scenario.
  • Sankey diagrams illustrated strong relationships between specific intents and topics (e.g., "GROW inform - habit or action" with "Nutrition") and between intents and entities (e.g., "GROW inform" with "Nourishment" and "Actions").

Limitations and Future Work

  • Corpus Size: While valuable, the Spanish corpus is relatively small (4522 turns) for training complex machine learning models.
  • Annotation Complexity: Hierarchical, multimodal taxonomies, while powerful, present challenges for typical machine learning methods, requiring more sophisticated classification strategies.
  • Language and Cultural Differences: The current analysis is primarily based on Spanish dialogues. Future work includes completing annotations for other languages (French and Norwegian) to enable cross-cultural comparisons.
  • Automatic Labeling: The current annotation relies on human annotators. Future work will involve developing classification strategies to automatically label dialogues using the proposed taxonomy.
  • Transfer Learning: The availability of multilingual data opens avenues for investigating transfer learning strategies for dialogue classifiers.
  • Deeper Analysis: Further analysis is needed to correlate dialogue act labels with emotion annotations derived from speech and facial expressions.

Summary updated

Abstract

This paper presents a dialogue act taxonomy designed for the development of a conversational agent for elderly. The main goal of this conversational agent is to improve life quality of the user by means of coaching sessions in different topics. In contrast to other approaches such as task-oriented dialogue systems and chit-chat implementations, the agent should display a pro-active attitude, driving the conversation to reach a number of diverse coaching goals. Therefore, the main characteristic of the introduced dialogue act taxonomy is its capacity for supporting a communication based on the GROW model for coaching. In addition, the taxonomy has a hierarchical structure between the tags and it is multimodal. We use the taxonomy to annotate a Spanish dialogue corpus collected from a group of elder people. We also present a preliminary examination of the annotated corpus and discuss on the multiple possibilities it presents for further research.
BibTeX
@inproceedings{montenegro2019dialogue,
  author = {César Montenegro and Asier López-Zorrilla and Javier Mikel Olaso and Roberto Santana and Raquel Justo and José Antonio Lozano and María Inés Torres},
  title = {{A Dialogue-Act Taxonomy for a Virtual Coach Designed to Improve the Life of Elderly}},
  booktitle = {Multimodal Technol. Interact},
  year = {2019},
  url = {https://openreview.net/forum?id=pgMmz3xxfLE},
}
Figure