Utterance selection for speech acts in a cognitive tourguide scenario

Felix Putze, Tanja SchultzInterspeech 2010

TL;DR

This paper introduces a cognitive memory model to enhance spoken dialogue systems (SDS) in dynamic environments like a car tourguide. By explicitly modeling user memory and relevance, their system generates more natural and informative interactions, significantly improving user experience compared to baseline approaches.

What Problem Does This Paper Address?

Developing spoken dialogue systems (SDS) for dynamic, real-world applications like in-car tourguides presents a significant challenge. Unlike controlled environments, these scenarios involve users who are not fully attentive due to concurrent tasks (like driving) and constantly changing contexts. Traditional SDS often assume a perfect, attentive user and lack the ability to adapt to these dynamic conditions. This leads to interactions that can feel unnatural, irrelevant, or overwhelming. The core problem this paper addresses is how to make SDS more cognitively aware, specifically by modeling the user's memory and dynamically selecting utterances that are both relevant and interesting to the user in real-time. This is crucial for creating a seamless and successful interaction experience in demanding environments.

Methods

The paper's core contribution lies in integrating a cognitive memory model into a spoken dialogue system. This model is inspired by human memory, featuring a semantic network of knowledge chunks with activation values.

  • Memory Model: They employ a long-term memory (LTM) model where information chunks are nodes in a semantic network. Activation values represent the relevance and usefulness of these chunks. This model has two key components:
  • Base Activation: Reflects the frequency and recency of a chunk's stimulation.
  • Spreading Activation: Represents associative activation from related nodes.
  • Noise Component: A pragmatic mechanism to simulate less predictable cognitive processes.
  • Item Importance & Interest: To go beyond simple activation, they define:
  • Importance: The ratio of spreading activation to base activation. This highlights chunks activated through association but not directly presented, marking them as potentially novel and interesting.
  • Interest: A static component representing intrinsic user interest in certain information, modeled using a Bayesian network.
  • Utterance Selection Strategy: The memory model directly informs utterance selection for both the system and a simulated user.
  • System Utterance Selection: Aims to find utterances that target chunks with high spreading activation (relevant), low base activation (new), and high interest. This is framed as reducing the "information urge."
    • User Simulation: The simulated user aims to reduce the mismatch between its own memory and its belief about the system's model of its memory. It uses speech acts like ask_most_important and give_metainfo to cue the system about its memory state and interest levels.
  • Workload Modeling: A scalar variable WL models mental workload, influencing utterance complexity, activation spread, and the user's desire for appropriate interaction complexity.

Datasets

Not discussed in the paper. The authors mention a database of Points of Interest (POIs) for their tourguide scenario, containing information chunks and alternative utterances.

Key Insights and Ideas

  • Cognitive Modeling is Key for Dynamic SDS: The paper strongly suggests that explicitly modeling human cognitive processes, particularly memory, is essential for creating effective SDS in challenging, dynamic environments.
  • Beyond Simple Activation: The distinction between "importance" (driven by context and association) and "interest" (intrinsic user preference) is crucial for selecting truly relevant and engaging information.
  • Dual Role of Memory Model: The memory model serves not only to guide the system's output but also to inform the behavior of a simulated user, enabling more realistic and useful evaluations.
  • Workload as a Modulator: Mental workload is a significant factor that needs to be accounted for, influencing how users process information and how systems should adapt their communication style.

Key Results

An evaluation study with 15 human judges compared four interaction strategies:

  1. nSnM: No strategy, no memory (random).
  2. SnM: Strategy, no memory.
  3. SM: Strategy, memory (the proposed approach).
  4. HC: Handcrafted (gold standard).

The results showed a clear progression: nSnM < SnM < SM < HC.

  • The proposed SM strategy significantly outperformed the baseline SnM (strategy but no memory), indicating the critical role of the memory model.
  • The SM strategy achieved scores much closer to the handcrafted gold standard (HC) than to the baseline interactions, demonstrating its effectiveness in generating meaningful and natural interaction behavior.
  • Specifically, for the system and overall interaction quality (including naturalness), the gap between SM and HC was small.

Limitations and Future Work

  • Limited Utterance Variety: The authors note that to completely close the gap with the handcrafted interaction, more variety needs to be added to the behavior of both the system and the user.
  • Future Work: The paper implies that further refinement of utterance variety and potentially more sophisticated modeling of cognitive processes could lead to even more human-like interactions. The current implementation also uses a simplified approach for simulating user interest and the impact of workload.

Summary updated

BibTeX
@inproceedings{putze2010utterance,
  author = {Felix Putze and Tanja Schultz},
  title = {{Utterance selection for speech acts in a cognitive tourguide scenario}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure