In "Deterministic Parsing of Syntactic Non-fluencies," Donald Hindle proposes a robust, rule-based approach to one of natural language processing's messiest hurdles: spontaneous speech. While traditional parsers often fail when confronted with the "dreck" of human conversation—hesitations, false starts, and self-corrections—Hindle demonstrates that these non-fluencies are surprisingly systematic. By treating corrections as structured "expunctions" of repeated syntactic elements, he shows that a deterministic parser can navigate speech errors without the need for complex backtracking or probabilistic guessing.
The Challenge of Spontaneous Speech
Spontaneous speech is rarely a stream of perfect, well-formed sentences. We frequently interrupt ourselves, as in "I was— we were hungry." For a computational model, these interruptions create a "detection" problem (where does the error start?) and a "resolution" problem (what should be removed to fix it?). Hindle argues that these errors are not random deviations but follow a strict logic of "copying" that can be handled by a deterministic parser—one that makes irreversible decisions on the fly.
The Fidditch Framework
The system is built upon Fidditch, a deterministic parser based on Mitchell Marcus's 1980 model. Fidditch operates using two primary data structures: a push-down stack for incomplete syntactic nodes and a three-constituent "window" (or buffer) for nodes that have been fully parsed but not yet attached to the tree.
As shown in Figure 4, the parser maintains a clear distinction between the current active node on the stack and the upcoming constituents in the window. This structure is crucial for the "Editing Rules" Hindle introduces.
Four Rules for Self-Correction
Hindle identifies a phonetic "editing signal"—often an abrupt cut-off represented in transcripts as --—as the trigger for correction. When this signal is detected, the parser applies one of four editing rules:
- Surface Copy Editor: This rule looks for identical word strings on either side of the signal (e.g., "if they'd-- if they'd").
- Category Copy Editor: This matches syntactic types. If the parser sees two determiners separated by a signal, it deletes the first.
- Stack Copy Editor: The most sophisticated rule, it searches the stack for a node of the same type as the one currently in the buffer. If a match is found, the older, incomplete version is "expunged."
- Restarts: Triggered by lexical cues like "well" or "oh," this rule causes the parser to abandon the current sentence and start anew.
Figure 1 illustrates a complex stack state where multiple nodes are tracked. The "expunction" process simply removes the erroneous "copy," allowing the parser to proceed as if the mistake never occurred.
Results and Implications
Testing the system on over 20 hours of sociolinguistic interviews (1,512 sentences), Hindle found that 27% of sentences contained non-fluencies. Remarkably, his simple deterministic rules successfully resolved the vast majority of these cases. Surface copying and stack copying were the most frequent successful interventions, with only 3% of cases leading to parser failure.
Beyond technical utility, Hindle offers a provocative insight into language acquisition. He suggests that non-fluencies might actually serve as "evidence" for children learning language. When a speaker corrects "People-- there's a lot of people," they are essentially providing a hint that "People" and "there" occupy the same syntactic slot, highlighting equivalent word classes.
Conclusion
Hindle’s work proves that resolving speech errors does not require "general-purpose reasoning" or complex likelihood evaluations. Instead, the "messiness" of speech can be tamed by simple, deterministic rules operating on surface strings and basic syntactic categories. By viewing non-fluency as a rule-governed process of copying and expunction, we can build parsers that are as resilient as human listeners.
Summary updated