When humans speak spontaneously, they do not produce clean, grammatically structured text. Instead, natural speech is littered with conversational artifacts: filler words ("um", "uh"), repetitions ("the the"), false starts, self-corrections, and fragmented words.
While human listeners effortlessly filter out this linguistic noise to extract core meaning, automatic natural language processing (NLP) systems heavily struggle with it. As Piroska Lendvai demonstrated in Learning to Identify Fragmented Words in Spoken Discourse, these irregularities degrade the performance and robustness of downstream applications. This article explores why disfluencies disrupt language models, how researchers detect and correct them, and when these vocal "imperfections" transition from being noise to vital signals.
The Spoken-Written Language Gap
The fundamental challenge of processing spontaneous speech stems from a distribution shift. Most downstream NLP models—such as machine translation engines, abstractive summarizers, and coreference resolvers—are trained on clean, highly structured written corpora (like books, news articles, or Wikipedia). Verbatim speech transcripts violate the syntactic rules these models have learned.
This mismatch causes what is known as disfluency-induced processing degradation across several domains:
- Entity and Term Recognition: Adam Palanica et al. in Do you understand the words that are comin outta my mouth? Voice assistant comprehension of medication names found that when voice assistants process vocal artifacts literally, the noise disrupts the recognition of core technical terms within a query.
- Structural Analysis: In tasks like zero pronoun resolution, the presence of fillers and repetitions obscures the structural clues needed to link omitted pronouns to their referents (Qingyu Yin et al. in Zero Pronoun Resolution with Attention-based Neural Network). Similarly, unsupervised topic segmentation becomes highly error-prone because conversational interruptions break the semantic continuity of discussion blocks (Ke Wang et al. in M³Seg: A Maximum-Minimum Mutual Information Paradigm for Unsupervised Topic Segmentation in ASR Transcripts).
Framing the Solution: From Tagging to Translation
To bridge this gap, researchers initially treated disfluency correction as a sequence-tagging or decoding problem. The goal was to identify the exact span of the disfluency and remove it.
Early architectures utilized sequence classifiers or specialized decoders. For example, Sankaranarayanan Ananthakrishnan et al. developed a "variable span" approach in Variable Span disfluency detection in ASR transcripts to classify entire word sequences rather than tagging tokens in isolation. To capture complex self-corrections, Dario Bertero et al. combined Deep Neural Networks with Conditional Random Fields in A comparison between a DNN and a CRF disfluency detection and reconstruction system, while Xuancong Wang et al. introduced a multi-pass beam-search decoder in A Beam-Search Decoder for Disfluency Detection that combined discriminative and generative language models.
However, simple deletion often fails when a speaker changes their mind mid-sentence, leaving behind ungrammatical fragments. A more powerful paradigm reframes disfluency detection as a style translation problem: translating a text from "noisy spoken style" to "polished written style."
[Verbatim Transcript] ───► [Translation/Rewriting Engine] ───► [Clean Text]
"We should go to, uh, no, let's meet at the office." "Let's meet at the office."
Bowen Zhou 0006 and Yuqing Gao pioneered this perspective in A phrase-level machine translation approach for disfluency detection using weighted finite state transducers, using Weighted Finite State Transducers (WFSTs) to map disfluent phrases to clean alternatives. This was expanded by Graham Neubig et al. in A WFST-based log-linear framework for speaking-style transformation to handle not just deletions, but complex word substitutions and the insertion of missing grammatical particles.
With the advent of deep learning, this rewriting task shifted to Neural Machine Translation (NMT) and Pre-trained Language Models (PLMs). Eunah Cho et al. demonstrated in Multilingual Disfluency Removal using NMT that training a single NMT model on multiple languages simultaneously mitigates data scarcity. More recently, Junwei Liao et al. adapted encoder-decoder language models in Generating Human Readable Transcript for Automatic Speech Recognition with Pre-trained Language Model, showing that optimizing for readability and structure provides a more significant boost to human comprehension than minor acoustic improvements.
A Performance Multiplier for Downstream Tasks
Why insert a dedicated disfluency correction step instead of allowing downstream models to learn to ignore the noise? The answer lies in error propagation. When an NMT system encounters a repetition or filler word, it frequently attempts to translate it literally, destroying the semantic coherence of the target sentence.
Dedicated correction modules act as a performance multiplier across cascade systems:
- Machine Translation: Pre-processing transcripts to remove conversational noise prevents translation models from hallucinating or stumbling over meaningless artifacts. This pipeline approach has been shown to drastically improve translation quality across multiple languages (Holger Schwenk et al. in Improved machine translation of speech-to-text outputs; Hao Yang et al. in The HW-TSC Video Speech Translation System at IWSLT 2020; Wu Jing et al. in Improve Speech Translation Through Text Rewrite). Vineet Bhat et al. quantified this effect in DISCO: A Large Scale Human Annotated Corpus for Disfluency Correction in Indo-European Languages, showing that filtering out conversational noise before translation yields substantial, immediate gains in BLEU scores.
- Summarization and Dialogue: Raw transcripts from meetings or telehealth sessions suffer from low information density. Prasenjit Mitra et al. (Generating Abstractive Summaries from Meeting Transcripts) and Zhengyuan Liu et al. (Context Aggregation with Topic-focused Summarization for Personalized Medical Dialogue Generation) established that distilling long, noisy conversation histories into topic-focused summaries requires a deep contextual understanding to separate structural noise from intended content, a concept thoroughly detailed by Maximilian Bisani et al. in Automatic Editing in a Back-End Speech-to-Text System.
Low-Resource Adaptation and LLM Behaviors
A persistent bottleneck in this field is the cost of human annotation. Labeling the exact boundaries of false starts and repairs requires significant linguistic expertise. To scale these systems to low-resource languages, Preethi Jyothi et al. introduced an adversarial training framework called Seq-GAN-BERT in Adversarial Training for Low-Resource Disfluency Correction. By pairing a small pool of labeled data with a generator that creates synthetic disfluencies, they forced the correction model to become highly robust to realistic speech variations without requiring massive human-annotated datasets.
Interestingly, modern Large Language Models exhibit an implicit, emergent capacity for disfluency removal. As observed by Yukiya Hono et al. in Integrating Pre-Trained Speech and Language Models for End-to-End Speech Recognition, when an LLM is integrated into an end-to-end speech recognition system, its internal prior—trained on clean text—naturally biases it to drop fillers and repetitions during generation. While this behavior automatically enhances transcript readability, it introduces a unique challenge: it can conflict with tasks that require high-fidelity, verbatim transcriptions.
When Noise is Actually Signal
Given the widespread effort to eliminate disfluencies, it is tempting to view them purely as errors. However, a crucial caveat exists: how we say things matters as much as what we say.
In clinical and diagnostic contexts, disfluencies transition from being noise to vital behavioral biomarkers. Shahla Farzana et al. explicitly tested this boundary in How You Say It Matters: Measuring the Impact of Verbal Disfluency Tags on Automated Dementia Detection. They discovered that systematically removing verbal disfluencies from transcripts—whether using human gold-standard annotations or automated predictors—significantly hurt the performance of automated dementia detection models.
The frequency, duration, and placement of hesitations or self-corrections provide critical windows into cognitive load and neurological health. Consequently, the modern consensus treats disfluency correction not as a mandatory cleaning step, but as a strategic choice: a preprocessing step optimized for semantic comprehension, but a feature to be fiercely preserved when analyzing human behavior.
Key Foundational Work
Learning to Identify Fragmented Words in Spoken Discourse (2003) — contributed 1 concept elements to this direction.
Automatic speech summarization applied to English broadcast news speech (2002) — contributed 1 concept elements to this direction.