TL;DR
This paper introduces a novel dialogue system that enhances user experience by employing multiple specialized agents (domain, strategy, and context agents). This multi-agent approach aims to make the system's capabilities more transparent to the user, leading to more effective information retrieval and easier management of complex, multi-goal dialogues.
What Problem Does This Paper Address?
Current dialogue systems often struggle with complexity, particularly when dealing with multiple information domains or intricate user goals. Users can become confused by the system's opaque behavior, leading to misunderstandings about what information is accessible and how to best interact with the system. Specifically, the paper identifies three key problems:
- Cross-domain information retrieval: Users may incorrectly assume that information spanning multiple data sources can be retrieved with a single query, leading to frustration when the system cannot fulfill such requests.
- Strategy transparency: When a system has both sophisticated, goal-specific strategies and simpler, more general ones, users might misinterpret the system's capabilities, believing it possesses an "all-powerful" strategy that doesn't exist.
- Context management for multiple goals: Existing systems often manage only a single context, forcing users to manually track multiple goals and their associated information, which is cumbersome and error-prone.
Methods
The paper proposes a multi-agent dialogue system designed to address these issues by making the system's functionality more visible to the user. The core innovation is the introduction of three distinct types of agents that collaborate:
- Domain Agents: Each domain agent is responsible for information retrieval within a specific domain (e.g., travel, cinema, baseball). They handle basic interactions and define domain-specific retrieval conditions. By having separate agents for each domain, the system explicitly signals the boundaries between different information sources, preventing users from making cross-domain assumptions.
- Strategy Agents: These agents manage information retrieval based on specific dialogue strategies. They are designed to make users aware of the different strategies available and their suitability for particular tasks. This helps users understand that strategies are task-specific and not universally applicable.
- Context Agents: These agents are crucial for managing complex dialogues involving multiple goals. They allow users to easily compare information across different contexts, simplifying the process of dealing with multi-goal interactions.
These agents take turns and coordinate their actions based on the discourse situation, aiming to provide a more transparent and user-friendly interaction.
Datasets
The system is evaluated using four CD-ROMs containing information on:
- Sight-seeing in Japan
- Hotel information in Japan
- Japanese and foreign cinema
- Japanese professional baseball player information
These CD-ROMs are grouped into domains: CD-ROMs 1 and 2 form a "travel domain," CD-ROM 3 is a "cinema domain," and CD-ROM 4 is a "baseball domain." The paper does not describe the creation of new datasets but rather the utilization of existing information sources.
Key Insights and Ideas
- Transparency through specialization: The core idea is that by breaking down the dialogue system's functionality into specialized agents, the system's capabilities and limitations become more apparent to the user. This contrasts with monolithic systems where user confusion is more likely.
- Explicit domain boundaries: Domain agents serve as clear indicators of where one information source ends and another begins, directly addressing the problem of users assuming seamless cross-domain access.
- Strategy awareness: Strategy agents help users understand that different approaches are needed for different tasks, preventing the misconception of a single, all-encompassing strategy.
- Simplified multi-goal management: Context agents significantly reduce the cognitive load on users when dealing with multiple, intertwined goals, making complex dialogues more manageable.
Key Results
The paper presents two examinations to evaluate the effectiveness of the proposed multi-agent system compared to a single-agent (domain agent) baseline system.
- Examination 1 (Business trip hotel search): The multi-agent system (new system) required fewer turns and characters to achieve the goal compared to the old system. Notably, users who first experienced the new system then used the old system more efficiently, suggesting the new system helps users learn effective strategies.
- Examination 2 (Comparing Kanazawa and Sendai for sightseeing): For a goal involving comparing multiple pieces of information, the new system also showed advantages. While session times were comparable, the new system facilitated more retrieval results, indicating better performance in handling multi-goal comparisons.
Qualitative findings suggest that users were happier with the hotels found using the new system and that the new system is better at dealing with multiple goals.
Limitations and Future Work
- Limitations: The paper acknowledges that the system's effectiveness is evaluated with a small number of subjects (eight typists, novices to dialogue systems) who were given a brief practice session. The examinations were conducted on specific, relatively simple goals.
- Future Work: The authors suggest that by making the system's abilities more visible, users will better understand what the system can and cannot do. The experimental results indicate that the multi-agent system allows users to retrieve information effectively and achieve their goals more easily. Further research could explore more complex scenarios and a wider range of user expertise.
Summary updated