Term-Aware Commitment for Streaming Medical Speech Translation

Live speech translation systems show words before the speaker has finished a sentence. They then rewrite those words as more audio arrives, and the rewriting makes the screen flicker. Current fixes decide what to freeze using rules that know nothing about language. One rule always hides the last few words. Another only allows changes to the last few words. A third shows a word only once every candidate translation the system is still considering agrees on it. In general conversation these rules work well. In a clinic they fail in a specific, costly way. Medical terms are multi-word units such as "chronic obstructive pulmonary disease" or "deep vein thrombosis." A system that commits to a translation of "chronic obstructive…" before hearing the head noun shows a wrong or generic term. It must then either erase it in plain view or leave the error on screen. Holding everything back to guard against this adds delay to the many words that carry no such risk.

We propose to let a domain term lexicon decide when the display may commit. The lexicon is a bilingual list of medical terms stored as a prefix tree. The system freezes ordinary words as soon as they are produced. It holds back, and allows revision of, only those words that fall inside a medical term the lexicon says is still incomplete. The central bet is that flicker and serious errors in specialized speech cluster inside multi-word terms. If so, spending the revision budget only there should yield fewer visible erasures on terms and lower delay on everything else, compared with fixed windows or agreement rules. The lexicon check is a prefix-tree lookup per token, so it adds almost no decoding cost. Because the lexicon is a separate component, it can be built automatically for new languages and specialties. We test directly how much its noise hurts.

Core Claim

A model's own confidence is least reliable on rare domain terms. A compact domain-tuned translator can be confidently wrong on the first word of a term whose meaning depends on words not yet spoken. An external lexicon supplies structure the model's confidence lacks: the knowledge that the source words heard so far are the beginning of a known term that has not finished. We claim that committing at term boundaries, rather than at fixed offsets or at beam agreement, improves the joint trade-off between flicker, delay and terminology accuracy on medical speech. We also claim it does so at a decoding cost indistinguishable from append-only decoding. The claim is falsifiable. If an uncertainty-only adaptive rule matches it, the lexicon contributes nothing, and the experiments include that rule as the decisive ablation.

Relation to Prior Work

Three papers on streaming display stability supply the machinery we build on and define the competitors. Re-Translation Strategies For Long Form, Simultaneous, Spoken Language Translation re-decodes the growing input and penalizes hypotheses that diverge from the displayed text. It also hides the trailing $k$ tokens. The authors concede that the hiding rule is blunt and pays the same delay whatever the content. Improving Stability in Simultaneous Speech Translation: A Revision-Controllable Decoding Approach limits revisions to the last $N$ tokens. Navigating the Minefield of MT Beam Search in Cascaded Streaming Speech Translation commits only the common prefix of all beam hypotheses and caches attention states to cut recomputation. All three were evaluated on general-domain speech (TED, CoVoST 2, conversational English-Spanish). None consults any lexical resource. We keep their caching and their append-only mode as the default behavior, and we replace their position-based or agreement-based commitment with a span-based one.

The specialization literature motivates the choice of model and exposes the gap. Enhancing medical coding efficiency through domain-specific fine-tuned large language models shows that a 1B model trained on ICD-10 code-description pairs reaches 69.20% exact match on MIMIC-IV discharge summaries. It still makes "hierarchy-level" errors, picking a term slightly too broad or too narrow. That is exactly the error a premature streaming commitment would produce. The paper works only on complete documents. Real-time Speech Summarization for Medical Conversations avoids flicker in clinical speech by emitting summaries every 30 seconds. It reports that quality degrades at 28.8% speech-recognition error rate and that raw large-model outputs hallucinate 25% of the time. It constrains our design: it rules out coarse batching for live subtitles, and it warns that noisy recognition will create false term matches, which we must measure.

The lexical-resource literature supplies the method for building the lexicon cheaply and sets realistic noise levels. Cross-Lingual Bootstrapping of Semantic Lexicons: The Case of FrameNet projects lexicon entries across languages through word alignments. It reports precision of 30–35% unfiltered and up to 68% (German) and 65% (French) after filtering. We reuse this projection to induce target-side term lists, and we use these precision figures to set the noise levels in our robustness study. LM-Lexicon: Improving Definition Modeling via Harmonizing Semantic Experts shows that domain-partitioned compact experts can beat frontier models on lexical tasks. It is offline and English-only, and we do not adopt its expert-mixing architecture. Simul-MuST-C attacks instability at training time with monotonic translations. It is complementary, and we treat it as orthogonal rather than competing.

Research Program

The system is a cascade. A streaming speech recognizer feeds a compact domain-specialized translation model, fine-tuned in two phases on term pairs and then on synthetic clinical variations, following the medical coding paper's recipe. The model decodes with cached attention states.

Alongside it runs a bilingual term lexicon stored as two prefix trees, one for source terms and one for target terms. At each step, the source tree marks any suffix of the recognized transcript that is a proper prefix of a lexicon term as an "open span." The target tree marks any suffix of the hypothesis that is a proper prefix of a target term. Let $o_t$ be the index of the first target token aligned, by the model's cross-attention or by the target tree, to an open span at step $t$. The display commits every token before $o_t$ immediately and never revises it, exactly like the revision-free mode. Tokens from $o_t$ onward stay hidden, or are shown greyed out in a display variant we test separately, and may be revised freely. A span closes when the source tree reaches a terminal node, or when the next recognized word cannot extend any term. To bound delay, a span that stays open longer than $D$ seconds is force-committed. $D$ is the only tuned parameter of the policy.

The research has three parts. First, an analysis that tests the premise directly. On unconstrained re-translation of medical speech, we measure what fraction of all erased characters falls inside lexicon term spans, relative to the fraction of text those spans occupy. If erasures are not concentrated in terms, the project stops there. Second, the commitment policy itself, compared against the baselines below under matched delay. Third, a scalability study. We replace the curated lexicon with automatically projected ones at controlled precision (roughly 35%, 50% and 68%, matching the projection results above) and at controlled coverage. We also run under realistic recognition noise rather than gold transcripts. This study shows how cheaply the method transfers to languages and specialties without curated bilingual terminologies.

Evaluation follows the critique in How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?, which finds that roughly 92% of studies use human-segmented audio and ignore compute time. We therefore run on continuous, unsegmented audio and report delay measured in wall-clock time on fixed hardware.

Data and Metrics

The file lists no medical simultaneous-translation benchmark. We found no reported result on Lacuna for streaming translation of medical speech, so we must build the domain test set. We use the real consultation audio from VietMed-Sum (Real-time Speech Summarization for Medical Conversations). Professional translators will translate a held-out portion of its transcripts into English, with medical terms marked. The English side of the lexicon comes from ICD-10-CM code descriptions, the 74,260 pairs used in the medical coding paper. The Vietnamese side comes from two sources: a curated version for an upper bound, and an alignment-projected version for the scalability study. For a general-domain regression check, we use CoVoST 2 German-to-English (CoVoST 2) and the IWSLT English-German TED setup.

The metrics are BLEU; normalized erasure (characters deleted from the display per output character); wall-clock average lag; and three term-level measures. Term accuracy is the exact match of reference terms in the final output. Term erasure is erasure restricted to term spans. Term display error is the fraction of terms that ever appeared on screen in a wrong form.

Baselines

The do-nothing reference is unconstrained re-translation with the same domain model. It gives the best final quality, and its flicker sets the ceiling. The opposite pole is append-only decoding (revision window zero, from the revision-controllable paper). On CoVoST 2 German-to-English it reports a normalized erasure of 0.00 and delay falling from 1057ms to 689ms, and it is the latency floor our policy must approach on non-term text.

The strongest published competitors are three. The bounded revision window with $N=3$ reports 26.33 BLEU at a normalized erasure of 0.11, down from 1.49, on CoVoST 2 German-to-English. Biased beam search with mask-$k$ reports erasure falling from 2.11 to 0.12 within 0.3 BLEU on IWSLT English-German. Beam-agreement emission with caching reports 1 BLEU over greedy decoding, more than 20% less character flicker and 40% less CPU than re-translation on internal English-Spanish data.

Beam agreement is the most dangerous of these, because it is already adaptive to content. Every baseline runs with the identical domain-tuned model, recognizer, cache implementation and hardware. Each gets the same tuning budget (a sweep over $k$, $N$, beam width or our $D$ on a development split), and we compare at matched wall-clock delay by tracing full delay-versus-erasure curves. Beating them means a lower term display error and term erasure at equal or lower delay, with BLEU and term accuracy within noise of unconstrained re-translation. We would also need to reproduce their reported numbers on CoVoST 2 and IWSLT to within noise, to show we lose nothing on general speech.

Two ablations isolate the lexicon's contribution. The first is an uncertainty-gated policy: it holds tokens whose beam disagreement or predictive entropy exceeds a threshold, tuned to the same delay. The second is a random-span policy, which holds spans with the same length distribution as lexicon terms but at random positions.

Why not just…?

Why not just gate commitment on the model's own uncertainty instead of a lexicon? This is the real threat. Beam agreement already does a version of it, and if it matches us the lexicon is unnecessary. Our case rests on overconfidence: a domain model can agree with itself across the whole beam on a generic rendering of an unfinished term. The hierarchy-level errors reported in Enhancing medical coding efficiency through domain-specific fine-tuned large language models suggest this happens, but the file contains no streaming evidence. The uncertainty-gated ablation at matched delay settles it. We commit in advance to reporting the outcome, and to recasting the work as a lexicon-informed uncertainty signal if uncertainty alone wins.

Why not just fine-tune a domain model and let its better terminology reduce flicker on its own? Domain fine-tuning improves final term accuracy, but nothing in the file shows that it reduces mid-sentence revisions. A correct final term can still be preceded by a wrong partial one on screen. Every baseline in our comparison already uses the specialized model. The premise analysis, which measures erasure concentration in term spans under that model, answers this directly.

Why not just inject the glossary through prompting or constrained decoding? Constraints act on what is eventually generated, not on when it is shown, so flicker inside a partially matched term remains. Multi-word constraints also widen the search, which conflicts with real-time budgets. Enhancing medical coding efficiency through domain-specific fine-tuned large language models further found prompting insufficient on large ontologies, with general models below 1% exact match. We include a glossary-prompted variant under the same emission rules to measure both its term accuracy and its wall-clock cost.

Decisive Checks

Three results decide the project. The first is the premise test. If term spans hold no more than their proportional share of erasures under unconstrained re-translation, the idea is wrong. The second is the matched-delay comparison against the uncertainty-gated and beam-agreement policies on the VietMed translation set. We need a clear reduction in term display error at no added delay. A null result here kills the lexicon's role. The third is the scalability curve. If term-aware commitment with a projected lexicon at around 50–68% precision, under real recognition noise, keeps most of its gain over beam agreement, the method transfers cheaply to new languages. If false term matches from recognition errors inflate delay past the append-only floor, it does not, and the curve will show at what lexicon quality the method stops paying off.