Whose Talks Can a Scientific Speech Recognizer Hear?

Speech recognizers adapted to scientific talks learn their technical vocabulary from written text, usually abstracts or papers pulled from one bibliographic database. Every database leaves things out. Indexes such as Web of Science, Scopus and PubMed favor English-language journals and Western publishers. So a recognizer adapted on one of them should learn the jargon of well-indexed research communities and miss the terms, place names, method names and local entities of researchers working outside them. Those researchers are often also non-native English speakers, which compounds the problem. We propose to measure this "indexing-to-recognition" bias directly. We will also test whether choosing adaptation text from several sources, specifically to fill vocabulary gaps, closes it. The measure is not word error rate alone. It is whether the scientific claims in a talk survive transcription, scored by a calibrated language-model judge whose agreement with human annotators is itself tested.

This matters because spoken science (conference talks, workshop presentations, recorded seminars) is where many results first circulate, and it is the natural next input for automated literature synthesis. If the speech front end silently drops the talks of under-indexed communities, synthesis systems built on top of it will repeat the exclusions that bibliometricians already document for single-database text pipelines.

Core Claim

Text-only domain adaptation is the cheap way to specialize a speech recognizer that pairs a speech encoder with a large language model. Its adaptation text has a source, and that source is a design choice with distributional consequences. We claim three things. First, per-talk technical-term errors track how well the adaptation text covers that talk's terms. Second, when the adaptation text comes from a single index, that coverage is systematically lower for speakers from regions the index under-represents. Third, choosing text from several databases in a coverage-aware way, at a fixed text budget, narrows the regional gap more than adding the same amount of single-source text or a random multi-source mix.

We expect part of the remaining gap to be acoustic rather than lexical. Text cannot fix that part. We quantify it separately rather than claim that text solves everything.

Where This Sits

The adaptation method we reuse is Text-only adaptation in LLM-based ASR through text denoising. It adapts a recognizer to a new domain without target audio by training the language model to clean up noisy text. To avoid breaking the link between audio and text, it mixes in source audio-text pairs, and it reports relative error gains of up to 22.1% on call-center data and out-of-domain improvements on SlideSpeech. That work treats the target-domain text as given. We make where the text comes from the experimental variable.

The strongest competing mechanism uses context local to each talk. SlideSpeech supplies more than 1,000 hours of slide-accompanied talks across 22 domains and biases recognition toward keywords read from the slides by optical character recognition. MaLa-ASR feeds the same keywords into the prompt of a frozen language-model decoder and reports word error rate dropping from 21.22% to 11.26% on SlideSpeech S95. Neither paper connects to any external scholarly source, and neither looks at which speakers benefit.

“Sorry, I Didn't Catch That” constrains our design. Even a prompt containing every target entity left a 24% failure rate on accented named entities. Fine-tuning on fewer than 1,000 synthetic accented samples gave gains of nearly 60%. This is evidence that text-side fixes have a ceiling. We therefore build in an acoustic arm that reuses its trick: a multilingual text-to-speech engine speaks the term inside a foreign-language carrier phrase.

The bias side is motivated by bibliometric evidence that single-database studies exclude non-English and regional work, in A bibliometric analysis of global research output on network meta-analysis, Panaroma of microglia in traumatic brain injury and Green economy studies amongst the global climate change challenge. It is sharpened by AtlasNLP, which found 79.2% of country-task pairs empty in the ACL Anthology. AtlasNLP also warns us about labels: 74% of its records had no documented country. We cannot infer speaker region from metadata automatically, so we will annotate it by hand.

Two synthesis evaluations show the downstream stakes. LitReview Arena draws its topics from OpenAlex alone. Automating Cognitive Distillation for Expert-Level Scientific Literature Synthesis builds its synthesis on a citation-defined core of papers. Both work only with written text. Enabling Inclusive Systematic Reviews uses a language-model judge to bring medRxiv preprints past the indexing lag of PubMed. That is the closest existing attempt to correct single-source bias with a judge, but it stays with written abstracts and treats acceptance by a top-quartile journal as ground truth. None of these touches speech.

Research Program

Evaluation sets. We start with SlideSpeech and add two annotations. The first is each speaker's institutional region, labeled from affiliations shown on slides or venue pages. The second is a native/non-native accent judgment from two annotators. Because SlideSpeech's sources may themselves skew toward well-indexed venues, we also collect a held-out set of roughly 50 hours of recorded talks from regional and non-English-hosted workshops. These are delivered in English and transcribed by hand. For each talk, annotators mark the technical terms and write down the two to five central claims.

Adaptation sources. We build adaptation text pools from OpenAlex, PubMed, arXiv, and regional or non-English indexes, using each source's abstracts in the matching domains. We cap every condition at the same token budget. For each talk domain we measure term coverage as the fraction of annotated terms that appear at least a minimum number of times in the pool.

The coverage-aware selector works greedily. It adds documents from any source in order of how many still-uncovered domain terms they contribute. Those terms come from a candidate term list mined from all sources together, never from the test talks themselves. We then run the text-denoising recipe on a WavLM-plus-Vicuna recognizer of the MaLa-ASR type, keeping the published mixing of source audio anchors to avoid misalignment.

Acoustic arm. For terms that remain wrong after text adaptation, we synthesize accented pronunciations from the same multi-source term list using multilingual carrier phrases. We then fine-tune only the projector, the small trainable module between encoder and decoder. This lets us split the remaining regional gap into a part that text closes and a part that only audio closes.

Claim-fidelity judge. We build a claim-fidelity judge modeled on LitJudge's in-context calibration from LitReview Arena. It receives the gold claims and a candidate transcript and decides, claim by claim, whether each one survives intact, is distorted, or is lost. The calibration examples include deliberately disfluent but correct transcripts. They target the "fluency trap" LitReview Arena documented, in which an uncalibrated judge rewarded polish over substance. We validate the judge against human claim-survival labels on a stratified sample of 1,000 claims, and we report agreement separately for native and non-native speakers. A judge that is less accurate on accented speech would hide exactly the bias we are trying to measure.

Baselines

The naive reference is the frozen recognizer prompted without keywords: the MaLa-ASR setup without slide keywords, which MaLa-ASR reports at 21.22% word error rate on S95. Every result is reported as overall error, biased word error rate on technical terms, and claim survival, each broken down by speaker region and accent group.

The strongest existing method is MaLa-ASR's slide-keyword prompting, reported at 11.26% word error rate on S95. We reproduce it with the same encoder, decoder and 15.7M-parameter projector. On talks without usable slides, and on our regional set where we expect slide extraction to be noisier, it gets keywords only from whatever slides exist. Beating it means a lower regional gap in term error and claim survival when the two methods are combined than with slide prompting alone, with overall error no worse. If database adaptation adds nothing on top of slides, that is a reportable negative result.

The second strong baseline is text-denoising adaptation on OpenAlex text alone at the same token budget, with the same hyperparameter search of 20 configurations per condition. The paper reports SlideSpeech gains but gives no specific figure; we found no reported number on Lacuna for its SlideSpeech word error rate.

Two ablations isolate what is new. The first is a random multi-source mix at the same budget, which separates coverage-aware selection from simply using more sources. The second is a single-source pool scaled to ten times the budget, which tests whether coverage is just a matter of size.

For the acoustic arm, we compare against the projector fine-tuning recipe of "Sorry, I Didn't Catch That", applied to single-source terms. On the US/SF Streets entity sets that paper reports a 44% baseline error, a 24% failure rate with perfect context, and nearly 60% accuracy gain from synthetic fine-tuning. We use those sets as a sanity check that our reimplementation reproduces the effect before we apply it to scientific terms.

Why not just…?

Why not just prompt with slide keywords and skip database adaptation? This is the most serious threat. Slides list exactly the terms of the talk, wherever its authors' work is indexed. Two pieces of evidence suggest it is not enough. Perfect-context prompting still failed on 24% of accented entities in "Sorry, I Didn't Catch That". SlideSpeech and MaLa-ASR both report that slide reading breaks down on cluttered or low-resolution slides. Neither, though, measured any effect by region. The head-to-head and combined conditions above settle the question directly. If slide prompting closes the regional gap by itself, we narrow the contribution to talks without slides and to spoken Q&A.

Why not just use a bigger single database? OpenAlex is enormous, so a skeptic would expect coverage to come from scale. The bibliometric studies cited above argue that what a database excludes is structural rather than a matter of sampling. The ten-times single-source ablation tests this. If it matches coverage-aware multi-source selection on the regional gap, our claim about where text comes from fails.

Why not just measure biased word error rate instead of building a judge? The judge is justified only if it adds information. "Sorry, I Didn't Catch That" shows a transcript can reach acceptable word error rate while getting the critical entity wrong, but entity-level error rate may already capture that. We will compute how well biased word error rate and the judge each predict human claim-survival labels. If biased word error rate predicts as well, we report it and drop the judge from the main results.

Why not just correct zero-shot transcripts with a frontier language model afterwards? Post-correction can only fix errors that a language model can reconstruct from the surrounding text. That is the same text-side ceiling the 24% perfect-context failure exposes. We include this as an extra condition with the same term lists provided to the corrector.

Decisive Checks

The central test is a per-talk regression of term error on adaptation-text coverage, with speaker region and accent as covariates. If coverage explains a significant share of the regional term-error gap under single-source adaptation, and coverage-aware selection shrinks that gap by more than the random-mix and scaled single-source ablations at equal budget, the claim stands.

A second check splits the gap. We report what fraction of the regional term-error gap remains after the best text-only condition and how much of that the synthetic accented-audio arm removes. If text closes almost none of it, the bias is acoustic, not about indexing. That would redirect the work, and we would say so.

A third check confirms the bias reaches synthesis. We extract claims from each condition's transcripts and count how often talks by under-indexed speakers lose at least one central claim. This tests whether the speech front end feeds the single-source exclusion into downstream literature tools. The judge's claim-level agreement with humans must be within five percentage points between native and non-native speakers before any of its claim-survival numbers are reported.