TL;DR
This paper introduces a novel annotation scheme and a supporting vertical alignment method to systematically capture complex disfluencies, particularly those common in stuttered speech. This approach allows for detailed, word-level analysis, crucial for building sophisticated language models that can better understand and process disfluent speech.
What Problem Does This Paper Address?
Understanding and modeling disfluencies in speech is vital for applications like speech recognition and, importantly, for developing tools to assist individuals who stutter. Previous annotation schemes often struggle to capture the complexity of disfluencies, especially multi-iteration repetitions and overlapping disfluencies, which are prevalent in stuttered speech. These schemes also sometimes lack the word-level granularity needed for building robust language models. The authors aim to bridge this gap by providing a comprehensive, consistent, and parsimonious annotation scheme that can handle intricate disfluency patterns at the word level.
Methods
The core contribution is a new annotation scheme that extends the "repair-structure" approach. It breaks down disfluencies into reparanda (the part of speech replaced), interruption points, optional editing terms, and alterations. What's novel is its ability to handle complex, multi-iteration, and overlapping disfluencies without requiring strict nesting, which simplifies annotation and analysis.
To address the difficulty of applying this scheme, the authors developed a Vertical Alignment Method. This method involves:
- Formatting: Transcribing words and starting new lines at interruption points.
- Vertical Alignment: Aligning words that are replacements for each other across different lines.
- Backtracking Extent: Marking how far back the speaker backtracked to start each new line (defining the reparandum).
- Annotation: Assigning disfluency indices to words based on their alignment and backtracking extent.
This visual, pen-and-paper (or tool-assisted) approach simplifies the identification of reparanda and their relationships, even in complex overlapping scenarios. The scheme also incorporates specific codes for revisions, editing terms (like "um"), and starters (words used to help restart phonation), providing a rich representation of speech disfluencies.
Datasets
Not discussed in the paper. The paper focuses on the annotation scheme and method, using example utterances to illustrate its application rather than a formal dataset for training or evaluation.
Key Insights and Ideas
- Handling Complexity: The scheme's ability to manage overlapping and multi-iteration disfluencies without strict nesting is a significant improvement, making it more practical for real-world speech, especially stuttered speech.
- Word-Level Granularity: The emphasis on word-level annotations is crucial for developing sophisticated language models that can learn patterns within disfluencies, rather than just identifying their presence.
- Visualizing Disfluencies: The Vertical Alignment Method provides an intuitive, visual way to understand and annotate complex disfluencies, making the annotation process more manageable for humans.
- Unified Treatment: The scheme attempts to unify the treatment of various disfluency types (repetitions, revisions, editing terms, starters), leading to a more consistent annotation framework.
Key Results
The paper demonstrates the effectiveness of the annotation scheme and the Vertical Alignment Method through numerous examples. It shows how the scheme can accurately capture:
- Multi-iteration repetitions (e.g., "can can can").
- Overlapping disfluencies (e.g., "a total of of total of seven hours").
- Revisions with omitted or inserted words.
- The role of editing terms and starters.
While no quantitative results are presented, the qualitative examples illustrate the scheme's power in detailing the structure and extent of complex disfluencies.
Limitations and Future Work
- Annotation Difficulty: The authors acknowledge that determining the annotation codes can still be difficult, even with the vertical alignment method.
- Future Work: They plan to develop a graphical computer tool to further simplify the annotation process, where the annotator aligns words, and the tool automatically assigns tags. This would streamline the process and potentially improve consistency.
Summary updated