TL;DR
This paper presents a two-stage approach to robust dependency parsing for spoken language understanding (SLU) of spontaneous speech. It leverages generic French linguistic resources for initial parsing and then employs a domain-specific reranker to select the most accurate syntactic analyses, significantly reducing the need for extensive in-domain annotated data.
What Problem Does This Paper Address?
Understanding spontaneous spoken language is challenging for several reasons. First, Automatic Speech Recognition (ASR) systems produce transcriptions with errors, especially for informal speech. Second, spontaneous speech often deviates from written language norms, featuring disfluencies, repairs, and non-standard syntax, making it difficult for parsers trained on formal text to handle. Building a dedicated parser for each application is labor-intensive and requires significant expertise. This paper tackles the problem of creating robust dependency parsers for SLU systems that can handle ASR errors and the complexities of spontaneous speech while minimizing the annotation effort required for domain adaptation.
Methods
The proposed method uses a two-stage parsing process:
- Generic Syntactic Parsing: This stage takes ASR output, represented as a word lattice, and processes it through a series of modules:
- Lexical Module: Groups word sequences into complex grammatical units (e.g., "au dessus de").
- Part-of-Speech (POS) Tagger: Assigns POS tags to words, creating a POS tag lattice.
- Lemmatiser: Associates lemmas to word-POS pairs.
- Chunker: Groups POS tags into non-recursive, non-ambiguous chunks, producing a chunk lattice.
- Partial Dependency Parser: This is the core of the first stage. It aims to identify syntactic arguments (subjects, objects) of verbs. It leverages two key generic French linguistic resources:
- Dicovalence: A syntactic lexicon of French verbs providing subcategorization frames (how verbs expect arguments).
-
FXMG (French Tree Adjoining Grammar): A large-coverage grammar that provides syntactic variations for subcategorization frames. The parser combines these resources by mapping Dicovalence subcat frames to FXMG tree families. These are then transformed into finite state automata for efficient processing of the chunk lattice. The output is a set of dependency hypotheses for each verb.
-
Dependency Parse Reranking: This stage adapts the generic parser to a specific application domain. It uses a machine learning classifier (IcsiBoost, a boosting algorithm) trained on domain-specific data.
- The classifier learns to distinguish correct dependency analyses from incorrect ones among the hypotheses generated by the first stage.
- This reranking significantly improves the accuracy by filtering out less likely analyses based on domain-specific knowledge.
Datasets
- French MEDIA spoken dialogue corpus: This corpus was used for both training and evaluation. It consists of dialogues from a Wizard of Oz system simulating a tourist information and hotel booking service. It contains 1257 dialogues from 250 speakers (approximately 70 hours of speech).
- The corpus was split into training, development, and test sets.
- The evaluation focused on the target verb "réserver" (to book).
- French TreeBank: Used for estimating parameters of the POS tagger.
- French Tree Adjoining Grammar (FXMG) and Dicovalence: These are existing linguistic resources used as generic syntactic knowledge.
Key Insights and Ideas
- Leveraging Generic Resources for Robustness: The paper demonstrates that using rich, generic linguistic resources like Dicovalence and FXMG can provide a good baseline for parsing spontaneous speech, even with ASR errors.
- Two-Stage Approach for Domain Adaptation: The key innovation is the separation of generic parsing from domain-specific adaptation. By performing domain adaptation only at the reranking stage, the authors significantly reduce the amount of in-domain data and expertise required.
- Word Lattices for ASR Integration: Directly processing ASR word lattices, rather than just the best word string, allows the parser to consider multiple recognition hypotheses, improving robustness to ASR errors.
- Reranking as a Powerful Adaptation Mechanism: The success of the reranking stage highlights the importance of domain-specific knowledge for disambiguation. It shows that a relatively small amount of annotated data for reranking can yield substantial performance gains.
Key Results
The paper evaluates its method on the MEDIA corpus, focusing on the verb "réserver."
- Full Dependency Analysis:
- On human transcriptions (ref. trans.), the reranked system achieved an F-measure of 64.9%, compared to a baseline of 40.4%.
- On ASR transcriptions (ASR trans.), the reranked system achieved an F-measure of 49.2%, significantly outperforming the baseline of 31.6%.
- The "Oracle" results (perfect reranker) show an F-measure of 63.2% on ASR, indicating the potential for improvement.
- Constituent Level Analysis:
- On human transcriptions, the reranked system achieved an F-measure of 90.6%, compared to a baseline of 82.4%.
- On ASR transcriptions, the reranked system achieved an F-measure of 78.2%, compared to a baseline of 70.1%.
These results show that the reranking approach significantly improves performance, especially on ASR transcriptions, and that the method is robust enough to be integrated into SLU systems.
Limitations and Future Work
- Limitations:
- The evaluation of full dependency analyses is strict: any ASR error in a constituent leads to the entire hypothesis being considered erroneous.
- The generic parser's coverage is not perfect (90% Oracle accuracy on reference transcriptions), with the remaining 10% attributed to limitations in linguistic resources or the size of the n-best list.
- Future Work:
- Integrate more domain-specific data into the reranking classifier, such as a concept ontology, to filter hypotheses and add semantic constraints.
- Further explore the integration of ASR and syntactic parsing.
- Expand the set of target verbs beyond "réserver."
Summary updated