TL;DR
This paper introduces NPCEditor, a system designed to simplify the creation of virtual humans (VHs) capable of natural language dialogue. It leverages a novel cross-language information retrieval approach for robust language understanding, even with noisy speech recognition input, and provides a user-friendly interface for developers and administrators.
What Problem Does This Paper Address?
Creating believable and engaging virtual humans (VHs) that can converse naturally with users is a significant challenge. While AI and computer graphics have advanced, making these VHs accessible to non-experts and ensuring their language processing is reliable remains difficult. Existing systems often require deep AI expertise, and their language understanding can be brittle, especially when dealing with imperfect speech recognition. The authors aim to bridge this gap by providing a practical, robust, and user-friendly system for building the natural language processing (NLP) components of VHs, particularly for question-answering scenarios.
Methods
The core of NPCEditor's language processing lies in a novel text classification approach based on cross-language information retrieval (CLIR). Instead of traditional methods that match a user's question to known questions, NPCEditor directly compares the user's question to known answers. This is framed as translating the question from one "language" (user's utterance) to another "language" (the character's response).
Key techniques include:
- Statistical Language Models: Used to represent the content of both questions and answers.
- Kullback-Leibler (KL) Divergence: Employed to measure the difference between probability distributions of language models, serving as the matching criterion.
- Maximum Likelihood Estimation (MLE) with Jelinek-Mercer Smoothing: Used for estimating language model probabilities.
- Multi-field Data Structures: NPCEditor can incorporate non-lexical features (like gaze direction) alongside text, treating them as additional "words" in a multi-field data structure for classification. This allows for more context-aware responses.
- GUI-based Editor (NPCEditor): A user-friendly interface that abstracts away the complex mathematical underpinnings, allowing designers to specify utterances, link questions to answers, and train classifiers without needing deep AI knowledge.
Datasets
The paper mentions using a language database from the SGT Blackwell virtual human for evaluation. This database contained 1,261 questions and 60 answer classes. The authors also note that NPCEditor has been used with numerous other virtual characters, implying the creation of many domain-specific datasets.
Key Insights and Ideas
- Shifting the Paradigm: Question-to-Answer Matching: The most significant insight is the move from matching questions to questions to matching questions directly to answers. This allows the system to consider the appropriateness of an answer, not just its relevance to a specific question, which is crucial for character-based dialogue.
- Robustness to ASR Errors: The CLIR approach, by treating input as a "language" and using statistical models, proves to be more robust to the noise and errors introduced by Automatic Speech Recognition (ASR) systems compared to traditional methods.
- Democratizing VH Development: The NPCEditor's GUI-based interface is a key contribution, making the development of sophisticated VH dialogue systems accessible to a wider range of users, including designers and administrators, not just AI experts.
- Contextualizing Dialogue: The ability to incorporate non-lexical features (like gaze) as part of the input significantly enhances the potential for more nuanced and contextually appropriate interactions.
Key Results
- Classification Accuracy: NPCEditor's CLIR approach achieved 17% higher accuracy than a Support Vector Machine (SVM) baseline and also outperformed a baseline that compared questions to questions (SLM). These differences were statistically significant.
- Robustness: The system showed good robustness to ASR errors, with performance degradation only becoming significant when the Word Error Rate (WER) exceeded 60%.
- User Engagement: Evaluations in various applications (museums, training simulations, virtual worlds) indicate successful user engagement and believability of the virtual characters.
- Efficiency: The NPCEditor system has been used to create characters quickly, with one example showing a novice designer creating 30-40 responses for a "Clue" game character in just two afternoons.
Limitations and Future Work
- Domain Specificity: The system is primarily designed for limited-domain dialogue, particularly question-answering characters. While other dialogue forms are possible, they require pre-specified responses.
- ASR Error Threshold: While robust, performance does degrade with very high ASR error rates (above 60% WER).
- Future Work: The paper mentions the potential for NPCEditor to translate semantic representations into text and to interpret utterances in a semantic representation, suggesting further avenues for NLP integration. The authors also highlight the ongoing development of the Virtual Human Toolkit, which incorporates NPCEditor, indicating a commitment to broader accessibility and application.
Summary updated