A Spoken Dialogue Interface for TV Operations based on Data Collected by using WOZ Method

Jun Goto, Yeun-Bae Kim, Masaru Miyazaki, Kazuteru Komine, Noriyoshi UrataniACL 2003

TL;DR

This paper introduces a spoken dialogue interface for TV operations, developed using the Wizard-of-Oz (WOZ) method to collect natural language commands. The system aims to simplify complex TV operations in the face of increasing digital broadcasting options and connected devices, demonstrating a user-friendly approach to interacting with televisions.

What Problem Does This Paper Address?

The proliferation of digital broadcasting (like BS digital TV) and the increasing number of connected peripheral devices (DVD players, recorders, game consoles) have made operating televisions significantly more complex. Traditional interfaces, often relying on remote controls with numerous buttons, can be cumbersome, especially for elderly users. This complexity creates a need for more intuitive and accessible ways to control television functions, such as program selection, searching, and device operation. The authors address this by proposing and developing a spoken dialogue interface that allows users to interact with their TV using natural language.

Methods

The core of this research lies in the Wizard-of-Oz (WOZ) method for data collection. In this setup, a human operator (the "wizard") simulated the intelligent behavior of the TV interface. Subjects were instructed to speak naturally to the TV, believing it understood them, while the wizard interpreted their commands and provided appropriate responses via a speech synthesizer and manipulated the user interface. This allowed the researchers to gather realistic dialogue data for TV operations.

The developed system incorporates several key modules:

  • Robot Interface (IFR): A physical interface, designed as a stuffed animal, that houses a microphone and speaker. It serves as the user's direct point of interaction, conveying voice synthesis and activating speech recognition. The design aims to foster a sense of warmth and closeness.
  • Speech Recognition: Employs an algorithm for sequential, real-time recognition with a high accuracy rate (around 95% for news programs). It dynamically updates its dictionary with new program titles and names to handle the frequently changing vocabulary of TV operations. Error feedback is provided on-screen.
  • Dialogue Processing: This module infers user intent by analyzing spoken utterances. Given the constrained domain of TV operations, it leverages a pattern-based approach, combining specific categories (like program name, genre, time) with predicates to understand commands. This is more efficient than general natural language understanding.
  • Pattern Matching: A mechanism that uses defined patterns (with meta-characters like *, +, @) to parse input statements and extract relevant information (slots) for generating TV commands and database queries.

Datasets

The paper describes the collection of dialogue data using the WOZ method. This data was gathered from 20 subjects (10 men, 10 women) aged 24-31, each interacting with the system for 5 minutes. A total of 1,268 utterances were collected. This dataset is a novel contribution, specifically tailored for TV operation dialogues.

Key Insights and Ideas

  • WOZ is effective for realistic data collection: The WOZ method proved valuable in capturing natural user utterances for TV operations, revealing how users actually speak when interacting with such a system.
  • User utterances are highly structured: Despite the freedom of natural language, 83% of user utterances were direct requests to the TV, and 89% of these contained specific keywords (program title, genre, etc.). This suggests users tend to focus on actionable commands within the TV domain.
  • Underestimation of TV capability: Users often underestimated the TV's capabilities, leading them to omit utterances for functions they assumed were not possible.
  • Demand for spoken interfaces is high: A questionnaire revealed that over half of the subjects expressed a desire for a spoken dialogue interface for TV operations.
  • Embarrassment is a barrier: A significant reason for subjects not wanting a spoken interface was embarrassment, particularly when watching TV with family. This suggests that user experience and appropriate environmental setup can mitigate this.
  • Pattern-based dialogue processing is efficient for constrained domains: For TV operations, inferring intent from specific categories and predicates is a viable and efficient approach, avoiding the need for complex general NLU.

Key Results

  • High request rate: 83% of collected utterances were requests to the television.
  • High keyword inclusion: 89% of requests included specific keywords related to program information or commands.
  • Positive user feedback on ease of use: In a preliminary test, all subjects could access information on desired programs, and many found program selection easy, appreciating the lack of need to understand hierarchical data structures.
  • High demand for spoken interfaces: Over 50% of users indicated they would like to use a spoken dialogue interface.

Limitations and Future Work

  • Speech recognition challenges: While high accuracy was achieved, the paper acknowledges that achieving perfect results with current speech recognition technology remains difficult, especially with diverse accents and background noise.
  • Embarrassment factor: The reluctance of some users to speak commands aloud, especially in social settings, needs to be addressed. The authors suggest that user experience and environmental configuration can help reduce this.
  • Limited scope of WOZ test: The preliminary test involved only 5 subjects, and the authors are conducting more detailed experiments to further demonstrate usefulness and examine problem areas.
  • Future work: The authors plan to continue refining the system, conducting more extensive experiments to prove the usefulness of the spoken dialogue interface for television control, and to identify and address remaining problem areas.

Summary updated

BibTeX
@inproceedings{goto2003spoken,
  author = {Jun Goto and Yeun-Bae Kim and Masaru Miyazaki and Kazuteru Komine and Noriyoshi Uratani},
  title = {{A Spoken Dialogue Interface for TV Operations based on Data Collected by using WOZ Method}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2003},
}
Figure