TL;DR
This paper details the development of the VOYAGER system, an urban navigation and exploration tool that bridges the gap between speech recognition and spoken language understanding. By integrating a robust speech recognition system (SUMMIT) with a probabilistic natural language parser (TINA) and a domain-specific application, the researchers demonstrate a step towards more natural human-computer interaction via spoken language.
What Problem Does This Paper Address?
While speech recognition technology has advanced significantly, enabling computers to convert spoken words into text, true "spoken language understanding" remains a major challenge. The core problem is moving beyond simply transcribing speech to enabling computers to comprehend the meaning and intent behind utterances, especially in realistic, interactive scenarios. This involves handling the inherent variability of human speech (hesitations, false starts, accents) and integrating linguistic knowledge to interpret requests within a specific application domain. The authors aim to address this by developing a system that not only recognizes speech but also understands it well enough to perform actions within a defined task.
Methods
The VOYAGER system is built upon three key components:
- SUMMIT Speech Recognition System: This system first transforms the speech signal using an auditory model to identify acoustic landmarks. These are then organized into a dendrogram and mapped to phoneme hypotheses using pattern recognition algorithms. The lexicon is represented as pronunciation networks, and the Viterbi algorithm is used for lexical decoding. The paper mentions ongoing experiments with artificial neural networks for phonetic classification, showing performance improvements.
- TINA Natural Language System: TINA is a probabilistic parser designed for two primary functions: providing constraints to the speech recognizer and interpreting the meaning of the utterance for the back-end system. It employs an agenda-based control strategy and a grammar represented as a shared network structure. Constraint filters (syntactic and semantic) and probabilities acquired from training data are used to favor likely analyses.
- Control Strategy: The interface between SUMMIT and TINA uses an N-best algorithm. The speech recognizer proposes its top N sentence hypotheses, and TINA attempts to parse them. The system stops with the first hypothesis that TINA successfully parses, effectively using TINA as a filter. If parsing fails, a rejection message is issued. An A* search is used to generate subsequent hypotheses if the initial "best" one fails. A word-pair language model is also used to constrain the search.
Datasets
The system was trained and tested using a corpus of spontaneous speech.
- Training Set: Data from 70 speakers (equally divided between male and female) was used for training.
- Development Set: Data from 20 additional speakers was used for development.
- Test Set: Data from 10 subjects (5 male, 5 female) consisting of 485 utterances was used for final evaluation. The average number of words per sentence in the test set was 7.7.
Key Insights and Ideas
- Domain-Specific Applications are Crucial: Using a realistic application domain like urban navigation (VOYAGER) is vital for collecting naturalistic data, testing system performance rigorously, and translating spoken input into meaningful actions.
- Integrating Speech and Language is Key: The success of spoken language understanding hinges on the tight integration of speech recognition and natural language processing. TINA's role in constraining the speech recognizer's search is a prime example of this synergy.
- Handling Spontaneous Speech is a Major Hurdle: The system's performance is significantly impacted by the challenges of recognizing spontaneous speech, including errors, unknown words, and linguistic phenomena not covered by the models.
- Discourse Capabilities Enhance Interaction: Even simplistic discourse capabilities, such as maintaining context for pronominal reference and handling underspecified queries, significantly improve the naturalness and usability of the system.
Key Results
- VOYAGER generated an action for 51.7% of the sentences in the test set.
- 48.3% of sentences failed to generate a parse, with reasons including recognizer errors (20.3%), unknown words (17.2%), and linguistic/back-end issues (10.5%).
- Despite parsing failures, the system rarely failed to provide a response once a parse was generated, due to conscious design choices to constrain TINA.
- Of the sentences that did not parse correctly, only 13% generated an incorrect response; the rest resulted in the "I'm sorry but I didn't understand you" message.
- User evaluations by naive subjects indicated that about 80% of the system's responses were judged appropriate, with an additional 5% being verbose but correct.
- About 87% of user queries were judged to be reasonable.
Limitations and Future Work
- Limitations: The paper acknowledges that the system's performance is significantly limited by speech recognition errors and the inability to handle spontaneous speech phenomena effectively. The vocabulary size (350 words) and the word-pair language model's perplexity (72) also represent constraints. The discourse capabilities are described as "simplistic."
- Future Work: The authors plan to improve SUMMIT's phonetic recognition accuracy by incorporating context-dependent models. They also aim to investigate control strategies that more fully integrate the speech recognition and natural language components, moving towards a more seamless and intelligent interaction.
Summary updated