TL;DR
This paper introduces "Dico II+", an in-vehicle dialogue system that enhances multimodal interaction by combining flexible menu-based dialogue with a "speech cursor." This novel approach allows users to navigate menus and browse long lists using haptic input and spoken output, enabling hands-free and eyes-free operation crucial for driving.
What Problem Does This Paper Address?
Interacting with in-car systems poses a significant safety challenge, as drivers need to keep their eyes on the road and hands on the wheel. While voice control offers a solution, existing systems often fall short by requiring visual attention for tasks like browsing lists or confirming selections. This paper addresses the problem of enabling truly eyes-free and hands-free interaction with complex menu-based interfaces in cognitively demanding environments like cars. Traditional approaches to list browsing via voice are often slow and cumbersome, or they still necessitate looking at a screen for confirmation.
Methods
The core innovation is the "Speech Cursor," which integrates with the GoDiS dialogue manager and the Dico II+ demonstrator. The system uses a haptic menu navigation device (like a mouse or keyboard arrow keys) to move focus through menu items. As focus shifts, the system provides spoken feedback ("voice icon") for each item, allowing the user to understand their current selection without looking at the screen. This is combined with Multimodal Menu-based Dialogue (MMD), which allows users to freely switch between and combine speech and haptic input within and across utterances. MMD also supports parallel multimodality for output, meaning information is presented both audibly and visually, but users can choose to ignore the visual component.
Datasets
Not discussed in the paper.
Key Insights and Ideas
- Eyes-free browsing is crucial for safety: The paper highlights that for in-car systems, the ability to interact without looking at a screen is paramount for safety.
- Speech Cursor bridges the gap: The Speech Cursor effectively solves the problem of navigating long lists without visual input, a common bottleneck in existing voice-controlled systems.
- Flexibility in interaction: The combination of flexible dialogue management (GoDiS) and the Speech Cursor allows users to choose their preferred interaction style, seamlessly blending spoken commands with haptic navigation.
- MMD enables richer interaction: Multimodal Menu-based Dialogue allows for more natural and efficient interactions by permitting the combination of different input and output modalities.
Key Results
The paper describes a functional system, Dico II+, that demonstrates the proposed Speech Cursor and MMD approach. A walk-through of a sample interaction for playing music illustrates how a user can select a song from a list using only spoken commands and haptic navigation (simulated by [DOWN], [UP], [OK] inputs). The authors claim this provides a "powerful and user-friendly way of interacting with menu-based interfaces in cognitively demanding environments."
Limitations and Future Work
The paper mentions that the combination of flexible spoken dialogue systems and the voice cursor technology is patent pending. While the paper focuses on the concept and demonstration, specific quantitative evaluations of user performance (e.g., task completion time, error rates, user satisfaction) are not presented. Future work could involve more extensive user studies to validate the effectiveness and usability of the Speech Cursor in real-world driving scenarios.
Summary updated