Who Gets Filtered Out? Auditing Speech Corpus Pipelines for Clinical Speakers

Large speech datasets are no longer collected by hand. Automated pipelines take in thousands of hours of web or broadcast audio and decide which pieces to keep. They drop clips that are too quiet, too noisy, too narrow in bandwidth, or that disagree with an automatic transcript. These rules were tuned on typical speakers. Speech from people with motor speech disorders such as dysarthria is often quiet, breathy, slow and full of long pauses, which is exactly what the rules are built to reject. If so, the pipeline acts as a silent selection filter. It removes clinical speakers before any model is trained, and the resulting models fail on those speakers for reasons that no amount of later modeling work can see.

This note proposes to measure that filter directly and then repair it. The measurement passes dysarthric and control recordings, made under the same recording conditions, through faithful reimplementations of each pipeline stage and counts who survives. The repair is a filtering stage that scores the recording (noise floor, bandwidth, reverberation) separately from the speaker's production (loudness, rate, pauses), so that only the first can trigger rejection. The test is whether this retains clinical speech that standard filters discard, without letting in more genuinely degraded audio, and whether it narrows the recognition gap between clinical and control speakers downstream.

Core Claim

Each stage of current curation pipelines is known to struggle with atypical speech in isolation. What has not been measured is how much these stages, applied in sequence, bias the composition of the corpus they produce. Our claim is that this bias is large, that it grows with dysarthria severity, and that it sits mostly in a few stages: energy- and pause-sensitive voice activity detection (the module that decides which stretches of audio contain speech), and transcript-consistency rejection. We also claim the bias can be removed at a matched false-accept rate for degraded audio. Separating channel quality from production atypicality is the key ingredient. Relaxing thresholds uniformly cannot achieve the same thing, because it trades clinical retention for noise one-for-one.

Relation to Prior Work

Pipelines whose filters we audit. The pipeline papers supply the stages under test.

  • GLOBE discards segments by signal-to-noise ratio and spectral bandwidth before enhancement.
  • MinSpeech thresholds on signal energy and signal-to-noise ratio.
  • Emilia adds vocal separation and speaker diarization (splitting audio by who is speaking).
  • GigaSpeech slices long audio with multi-stage forced alignment, meaning it times each word against a known transcript, and checks for transcription errors.

None of these papers report how their filters treat atypical speakers. They are the objects of our audit, not competitors.

Clinical speech work that motivates the audit. Three papers show that the component parts break on dysarthric speech.

  • Yeung et al. show that standard forced alignment drifts on TORGO dysarthric speech. Median agreement starts at 61.7% and reaches 94.2% only with pause-aware anchors and lattice verification.
  • Feng et al. show that speaking-rate mismatch dominates recognition failure on UASpeech. They also show that generative voice conversion can add spectral artifacts that make things worse.
  • The TORGO acoustic-modeling study shows that standard modeling assumptions, such as frame shift and co-articulation context, are wrong for slow, reduced speech.

All three operate on already-curated clinical corpora. They repair recognition or alignment after the data exist. We instead ask what happens to clinical speech inside a large-scale pipeline, before any clinical corpus exists. We reuse Yeung et al.'s verification idea as one component, but change its role: their method fixes an alignment, while ours uses verification to decide whether to keep a segment.

The broader pattern. The clinical-generalizability literature documents the same failure in other settings: systems built on healthy cohorts degrade on patients. Examples include neurofeedback for sleepiness and insula downmodulation for pain. That work locates the gap in the model or the protocol. We test whether, for speech, part of the gap is created earlier, in the data-selection step.

Possible extension to speech neuroprostheses. Clinical speech neuroprostheses such as Metzger et al. decode into self-supervised speech units learned from large curated audio. If those corpora exclude atypical acoustics, the unit vocabulary itself may be skewed. We keep this as a secondary question rather than a central claim, because the file contains no evidence on it.

Research Program

Stage 1: the audit. We reimplement four filter families at published-style operating points:

  • signal-to-noise ratio and bandwidth thresholds (GLOBE-style);
  • energy thresholds plus voice activity detection (MinSpeech-style);
  • separation and diarization (Emilia-style);
  • alignment-consistency rejection, which drops segments where an automatic transcript disagrees with the reference (GigaSpeech-style).

We run them on TORGO and UASpeech. Both contain dysarthric speakers and controls recorded in the same setup. Any difference in survival between the groups can therefore be attributed to the speaker rather than the microphone, which is the property that makes the audit interpretable. We report per-stage and cumulative survival by severity grade, plus the acoustic profile of what is discarded: speaking rate, pause durations, intensity and pitch variability. We also measure whether the enhancement stage changes these acoustic properties more for dysarthric than for control speech, since Feng et al. found that synthetic processing artifacts hurt recognition.

Stage 2: the repair. The proposed filter makes three changes.

  1. It estimates channel quality only from non-speech intervals and from bandwidth. It does not use speech energy.
  2. It replaces fixed-silence segmentation with pause-tolerant segmentation, which allows long within-utterance pauses before cutting.
  3. It replaces outright rejection on transcript disagreement with Yeung-style lattice verification. Segments that disagree are kept but flagged, and they receive a verbatim transcript layer alongside the normalized one, following the dual-layer transcription design of IndicVoices.

Stage 3: downstream consequences. We train recognizers on corpora curated by each pipeline from the same mixed source pool. We then measure the clinical-versus-control error gap on held-out speakers.

Baselines

Naive reference: the standard cascade. This is the unmodified sequence of thresholds above, applied to the same pool. It is the default practice and fixes the starting clinical survival rate. For the alignment stage specifically, the strongest number on file is the off-the-shelf median agreement of 61.7% on TORGO dysarthric speech reported by Yeung et al..

Strongest existing method: the standard cascade with Yeung et al.'s three-stage aligner in place of the default aligner. Their reported 94.2% median agreement on TORGO is the best clinical alignment result in the file, so this baseline already contains the best known clinical fix. It is the competitor most likely to erase the effect we predict. We run it on identical audio with the same tolerance window. The proposal beats it only if it retains significantly more dysarthric segments at equal or better false-accept of degraded audio. If it matches it, the conclusion is that alignment was the whole problem.

Key ablation: uniform threshold relaxation. Every standard threshold is loosened until dysarthric survival equals that of our filter. We then compare how much synthetically degraded control speech each lets through. This isolates the channel-versus-production separation. If uniform relaxation matches our false-accept rate, the separation adds nothing.

Downstream baselines. For recognition, we re-run the recipe from the TORGO acoustic-modeling study: 15 ms frame shift, position-independent phones, a compact four-layer network. That paper reports a 17.62% average relative WER reduction over its baseline and per-speaker WER as low as 7.86%. We also apply time stretching as in Feng et al., who report UASpeech PER falling from 91.2% to 73.4%, against 56.8% for healthy controls. Both are applied identically on top of every curated corpus, with the same tuning budget. They are the modeling-side fixes our data-side fix must add to rather than duplicate.

Ingredients two and three of the repair also get their own ablations. We swap pause-tolerant segmentation back to the standard segmenter, and flag-and-keep back to reject, each with everything else held fixed.

Why not just…?

Why not just lower all the thresholds? This is the most serious threat, and it is the ablation above. Lowering every threshold admits more quiet, slow speech, but it also admits more genuinely noisy and band-limited audio, which is what the filters exist to remove. The decisive comparison is clinical retention at a fixed false-accept rate on degraded control audio. If uniform relaxation lies on the same curve as our filter, we will report that the fix is just a threshold choice and drop the channel-separation claim.

Why not just fine-tune on TORGO or UASpeech afterwards, so corpus composition does not matter? It may not matter, and Stage 3 tests this directly. We fine-tune models pretrained on standard-curated and repaired corpora on identical clinical data, then compare held-out-speaker error. If the gap vanishes after fine-tuning, the audit still has documentary value but the repair has no downstream benefit. We would say so. The case against relying on fine-tuning is that clinical fine-tuning sets are tiny; TORGO has eight dysarthric speakers. Prior gains there came from strong constraints on the model (TORGO study), which suggests pretraining data matters when adaptation data is this scarce. That remains a hypothesis.

Why not just use Yeung et al.'s aligner and stop? Their method addresses only the alignment stage. Energy thresholds and voice activity detection act before alignment and can discard clinical audio that never reaches the aligner. The per-stage audit shows how much loss happens upstream of alignment. If it is negligible, this alternative wins, and the strongest-baseline comparison will show it.

Why not just time-stretch clinical audio before filtering? Feng et al. needed a known target duration, which unprompted web audio does not provide. Stretching also changes the very cues a verbatim transcript should preserve. We nonetheless include rate-normalized filtering, using an estimated syllable rate, as a variant to check.

Proposed Experiments and Decisive Checks

The first decisive check is the audit result. If dysarthric survival through the standard cascade is not clearly below control survival on TORGO and UASpeech, and does not fall with severity, the premise fails and the project stops there. No survival-rate figures for these pipelines on clinical speech were found on Lacuna, so this is a genuinely open measurement.

The second check is the matched false-accept comparison against uniform relaxation and the Yeung-aligner cascade. For this, degraded audio is created by adding noise, band-limiting and adding reverberation to control utterances, which gives known positives.

The third check is downstream. We simulate web-scale curation by seeding a large general English pool, drawn from the same kind of sources as GigaSpeech, with TORGO and UASpeech speakers held out at the speaker level. We curate it with each pipeline, train identical recognizers, and report WER and PER for dysarthric and control test speakers, with and without clinical fine-tuning, with time stretching and the TORGO-style modeling recipe applied uniformly. The repair counts as successful only if it narrows the dysarthric-minus-control gap without raising control error.

A secondary analysis asks whether self-supervised speech units trained on each curated corpus encode dysarthric segments with higher reconstruction error. This bears on unit-based neuroprosthesis decoders, but it is reported as exploratory.

The main threat to validity is that seeding is artificial: real web audio contains atypical speakers who are never labeled. We address this by reporting the audit separately on unmodified clinical recordings, where the filter-bias finding stands without any simulation.