Speech-Plans: Generating Evaluative Responses in Spoken Dialogue

Marilyn A. Walker, Steve Whittaker, Amanda Stent, Preetam Maloor, Johanna D. Moore, Michael Johnston, Gunaranjan VasireddyINLG 2002 · OpenReview

Introduction

In spoken and multimodal dialogue systems, such as virtual assistants or restaurant recommenders, the "information presentation phase" is often the most frustrating part of the interaction. When a database query returns multiple valid options, systems typically present them sequentially (e.g., "Option 1 is X, Option 2 is Y..."). Because speech is transient, this sequential presentation places a massive memory burden on users, who must mentally track complex trade-offs between multiple options and their various attributes. This research introduces SPUR (Speech-Planning with Utilities for Restaurants), a module designed to abstract, highlight, and compare options to facilitate informed decision-making without cognitive overload.

User Modeling and Utility Scoring

The system addresses the cognitive bottleneck by adapting multi-attribute decision theory to natural language generation. The process begins with User Modeling, utilizing the SMARTER procedure to elicit preferences. Users rank domain attributes such as cost, food quality, and decor, which the system converts into mathematical weights.

Raw database values are then mapped to a 0-100 utility scale. The system calculates an overall weighted utility score for each option based on the specific user's model. This allows the system to distinguish between dispositional preferences (long-term biases) and situational constraints (current query parameters), ensuring that recommendations remain relevant to the user's general tastes even when searching for specific categories.

Table 1: Restaurant utilities and weighted attribute scores calculated for a specific user model.

The SPUR Module

The core contribution of this work is the SPUR module, which takes a user model, a set of database options, and a dialogue goal to generate three distinct types of speech plans:

  • Recommend: Identifies the highest utility option. It uses z-scores to find "outlier" attributes—those that are exceptionally good or bad for that specific user—and mentions only those attributes as justification, saving valuable audio time.
  • Compare: Selects the top options and highlights only the attributes that significantly differ among them, skipping attributes where the restaurants are similar or average.
  • Summarize: Provides a high-level overview of a selected set, such as stating the price range and the specific dimensions where the options vary.

System Integration and Results

SPUR was successfully integrated into MATCH (Multimodal Access To City Help), a working dialogue system deployed on a mobile Fujitsu PDA. This integration allows for multimodal interaction: for example, a user can circle a cluster of restaurants on a map with a stylus and simply say "compare," prompting the system to generate a tailored spoken comparison on the fly.

The MATCH system running on a mobile PDA, demonstrating the multimodal interface for restaurant selection.

Qualitative results demonstrate a stark contrast between SPUR and traditional baseline models. While a "No-Model" baseline verbosely lists all attributes for every restaurant, SPUR generates concise, 2-3 sentence responses that highlight only the most relevant key attributes. By mathematically defining what makes an option "stand out" for a specific user, the system significantly reduces the verbosity of the spoken response and the associated cognitive load.

Conclusion and Future Work

By leveraging multi-attribute decision theory, SPUR provides a framework for generating concise, user-tailored evaluative responses in spoken dialogue. While initial results are qualitatively superior to baseline systems, future work will focus on formal quantitative evaluations to measure efficiency gains and user satisfaction. Additionally, the authors plan to enhance the naturalness of the generated speech by interfacing SPUR with more advanced sentence planners.

Summary updated

BibTeX
@inproceedings{walker2002speech,
  author = {Marilyn A. Walker and Steve Whittaker and Amanda Stent and Preetam Maloor and Johanna D. Moore and Michael Johnston and Gunaranjan Vasireddy},
  title = {{Speech-Plans: Generating Evaluative Responses in Spoken Dialogue}},
  booktitle = {INLG},
  year = {2002},
  url = {https://openreview.net/forum?id=TlNGNy76ibr},
}
Figure