TL;DR
This paper introduces novel syntactic models for parsing spoken language with repairs, aiming to better represent the natural, interrupted flow of speech. By eliminating explicit "error" labels and focusing on the generative process, their REPAIRED-INT model achieves state-of-the-art accuracy in both overall parsing and the detection of repaired segments.
What Problem Does This Paper Address?
Spoken language is inherently messy, often featuring self-corrections or "repairs." These repairs, where a speaker interrupts themselves and restarts a phrase, pose a significant challenge for natural language processing systems, particularly parsers. Traditional parsing models, designed for clean, written text, struggle to accurately represent and process these disfluencies. Existing approaches to speech repair parsing often rely on explicit labels for "errors" (reparanda) and "corrections" (alterations), which can be linguistically unnatural and computationally inefficient. This paper tackles the problem of building more robust and linguistically motivated syntactic models that can parse speech containing repairs more effectively, by better reflecting how humans might process such interruptions in real-time.
Methods
The authors propose three new syntactic models for speech repair, building upon the intuition that speakers don't intend to produce errors but rather to produce fluent speech or corrections.
- Eliminating "Error" Labels: A key innovation is eschewing explicit labels for erroneous constituents (like "EDITED") during processing. Instead, they introduce a "REPAIRED" category to mark the end of a corrected section. This aligns with the idea that speakers stop speech upon noticing an error, rather than explicitly signaling it as an error.
- Right-Corner Transform: They leverage a right-corner transform, which converts standard phrase-structure trees into highly left-branching structures. This transformation is beneficial for speech repair as it naturally represents unfinished constituents and allows for a more sequential, left-to-right processing, mimicking human speech production.
- REPAIRED-INT Model: This is the core proposed model. It transforms the original repair structure by making the alteration the right-most child of the original "EDITED" constituent, removing the "EDITED" label, and adding a "REPAIRED" label. This approach aims to integrate repairs more seamlessly into the parse tree.
- Other Models (EDITED-BIN, REPAIRED-BIN): For comparison and to isolate the impact of different modifications, they also introduce EDITED-BIN (binarizing repairs and keeping the EDITED label) and REPAIRED-BIN (similar to REPAIRED-INT but with a slightly different structural transformation).
Datasets
The experiments were conducted using the Switchboard corpus. This corpus is a syntactically annotated collection of spontaneous dialogues. For training, sections 2 and 3 of the corpus were used, and for testing, section 4 (subsections 0 and 1) was employed. Data from the tail end of section 4 was used during development. The corpus was preprocessed by stripping punctuation, empty categories, typos, repair structure categories, and partial words to simulate the input a speech recognizer might produce.
Key Insights and Ideas
- The "Error" is in the Recognition, Not Production: The core insight is that speakers likely don't intend to produce errors. The "error" is only recognized as such when a correction begins. This shifts the modeling focus from explicitly labeling errors to modeling the process of correction and integration.
- Left-Branching Structures Aid Repair Parsing: The right-corner transform, which creates left-branching structures, is particularly well-suited for representing unfinished constituents in speech repairs. This structure allows for a more natural integration of repairs into the overall parse.
- REPAIRED-INT is the Most Promising: The REPAIRED-INT model, by eliminating explicit error labels and using a right-corner transform, achieves the best performance, suggesting that this approach better captures the linguistic reality of speech repairs.
- Implicit Repair Cues are Powerful: The success of models that implicitly handle repairs (like REPAIRED-INT) suggests that explicit repair markers might not be as crucial as previously thought, and that the structural context is more informative.
Key Results
The paper evaluates its models using a probabilistic CYK parser and reports results on two metrics: Parseval F-measure (overall parsing accuracy) and Edit-finding F-score (accuracy in recognizing edited words).
- REPAIRED-INT Achieves State-of-the-Art: The REPAIRED-INT model achieved the highest Edit-finding F-score (39.2) and a competitive Parseval F-measure (71.77).
- Significant Improvement over Baselines: REPAIRED-INT showed statistically significant improvements over previous work, including the "Standard Right Corner" baseline and the Hale et al. (2006) approach, particularly in recognizing edited words.
- EDITED-BIN Shows Promise: Interestingly, the EDITED-BIN approach, while slightly lowering overall Parseval F-measure, performed almost as well as REPAIRED-INT on the Edit-finding F-score, suggesting that even with explicit "EDITED" labels, structural modifications can improve repair detection.
Limitations and Future Work
- Assumption of Maximally Local Repairs: The REPAIRED-INT model, in its current form, relies on tracing back from a REPAIRED node to find its ancestor. This implicitly assumes repairs are "maximally local," which might not always be true. Future work aims to relax this assumption and explore how to handle repairs of varying distances.
- Implicit Incentive for Short Repairs: The current approach might implicitly incentivize shorter repairs, which is counter to the observation that most repairs are indeed short. Future work will investigate this further.
- Exploring the EDITED-BIN Insight: The strong performance of EDITED-BIN on edit detection, despite its structural differences, suggests there are elements of repair recognition that this modification captures. Future work will explore these elements further.
- Extensibility to Speech Recognition: The authors note that the left-to-right parsing architecture is well-suited for potential extension to speech recognition systems.
Summary updated