Speak Only What Has Been Checked: Verification-Aware Commit Policies for Streaming Spoken Answers

A voice assistant built on a large language model has two clocks running at once. The model writes text quickly, often tens of tokens per second. The speaker reads that text aloud slowly, at roughly two to three words per second. The gap between the two clocks is free time. A fact-checker could use that time to test a claim after the model has written it and before the speaker says it. This proposal asks how far that free time goes, and what the system should do in the moments when it runs out.

We propose a commit policy: a small controller that sits between the language model and an incremental text-to-speech engine. Incremental text-to-speech means a synthesizer that starts producing audio from the first few words instead of waiting for the full sentence. For each upcoming chunk of text, the controller decides among three actions. It can release the chunk to the synthesizer, hold it back with a natural filler or pause while a checker finishes, or, if a claim already spoken turns out to be false, splice in a short spoken self-correction. The central empirical question is whether this adaptive policy beats the obvious alternative of holding every word back by a fixed delay until it is verified. That fixed delay is a strong baseline, and the proposal is built so that it could win.

Core Claim

The two literatures each assume away the other's constraint. In-generation hallucination methods such as Monitoring Decoding check rolling chunks of partial text against a larger reference model and backtrack locally when the check fails. They treat the added latency (113.78 ms/token in that paper) as a text-throughput cost. They never ask whether a listener would notice it. Incremental speech systems, from INPRO_iSS to the neural prefix-to-prefix TTS, assume the incoming text is correct. They re-plan only when their buffer runs dry or when a user interrupts.

Our claim is that the speaking rate creates a structured latency budget, and that this budget is spent unevenly. It is scarce at the start of an answer, which is exactly where risky content tends to sit. It becomes abundant later in the answer. A policy that knows where the risky words are and how long checking them will take can therefore reach a lower rate of false claims actually spoken, at a lower delay before the first audio, than any fixed hold-back.

The budget can be written in one inequality. Suppose a claim ends at word position $p+L$, where $p$ is its start and $L$ its length. Let $r_g$ be the generation rate and $r_s$ the speaking rate, both in words per second. Let $v$ be the checker's latency on that claim, and $d$ the delay before the first audio. The claim is verified before its first word is spoken when

$$\frac{p+L}{r_g} + v \;\le\; d + \frac{p}{r_s}.$$

Because $r_g \gg r_s$, the right-hand side grows much faster than the left as $p$ increases. The binding constraints are therefore early claims and slow checkers, such as retrieval or knowledge-graph lookups. A fixed hold-back must set $d$ large enough for the worst early claim on every turn. An adaptive policy can instead start speaking a low-risk prefix, such as "The film was directed by…", and spend a filler or a lengthened pause only at the boundary before an unverified name. Hesitation stops being an engineering fallback, as it was in INPRO_iSS, and becomes a way of paying for verification exactly where the budget is short.

Research Program

We will build a pipeline with four parts. The first is a text language model streaming tokens. The second is a risk estimator that marks likely-fragile spans, typically named entities and numbers. The third is a checker. The fourth is an incremental synthesizer.

The risk estimator reuses two existing signals. One is the high-surprise "key tokens" idea from SH2, which uses tokens the model found hard to predict to trigger a contrastive re-decoding. We use the same tokens only as a cheap flag, not as a decoding change, because SH2's roughly 3x slowdown would eat the budget. The other is the small-model versus large-model disagreement score from Monitoring Decoding, which supports that paper's observation that only about 10% of tokens govern factuality.

We will run two kinds of checker, because their latency profiles differ. The first is the Monitoring Decoding reference-model check with its local tree revision. The second is a knowledge-graph check in the style of Neural Path Hunter. That paper uses a token-level critic to find hallucinated entity mentions and then searches nearby graph paths for a replacement, but it runs only after a whole dialogue turn is finished. We run its critic and retrieval per span, inside the budget.

The synthesizer is the prefix-to-prefix system of Ma et al., which pairs the Tacotron 2 spectrogram model with the Parallel WaveGAN vocoder under a lookahead-$k$ policy. Out of the box it assumes an immutable, growing text stream, so we extend it in two ways. First, it must accept a hold signal at word boundaries. Under a hold it emits either a silent pause or a filler token ("uh", "let me see"). Second, it must accept a repair signal, which inserts a correction phrase at the next word boundary using the chunk-splicing approach of Wester et al.. In that work, focus-group users preferred reactive re-planning over a system that ignored interruptions. Vowel prolongation, the other hesitation device studied in Synthesising Uncertainty, needs explicit duration control that Tacotron 2 lacks. We treat it as an optional extension on a duration-controllable synthesizer, not a core requirement.

The policy itself is deliberately simple, so that the comparison isolates the idea rather than a learning algorithm. At each word boundary it holds the next span when two conditions are both true: the span's risk exceeds a threshold, and the checker's predicted finish time falls after the span's scheduled audio start. Otherwise it releases the span. A span is repaired only when a released claim is later refuted. Sweeping the threshold traces a curve of spoken-hallucination rate against delay. We will also fit a cost-weighted version, with separate penalties for delay, stall time and uncorrected falsehoods, to report a single operating point.

One tension must be faced directly. Synthesising Uncertainty showed that filled pauses and prolongations lower listeners' perceived certainty. They did so even when every sentence already began with "I think", and the vocal-effort and disfluency effects interacted. Fillers are therefore not free. When they land before correct answers, they undermine trust in true content. When they land before claims that really are shaky, they act as honest signals. We will measure this in the listener study rather than assume either outcome. The "fluency trap" documented by the SHALLOW benchmark, where speech models produce polished output that flips meaning, is the reason we accept some loss of fluency in the first place.

Baselines

The naive reference point streams the language model's output directly into the prefix-to-prefix synthesizer with no checking at all. This gives the lowest achievable delay, which Ma et al. report as constant latency under 0.3 seconds with a lookahead-2 English Mean Opinion Score of 4.19, against 4.20 for full-sentence synthesis. It also gives the highest spoken-hallucination rate. Every other system is measured against both ends of this point.

The quality ceiling is verify-then-speak. The full answer is generated, corrected by Monitoring Decoding (or the Neural Path Hunter critic on dialogue data), and then synthesized. This approximates the best factuality available from existing methods. On factuality, Monitoring Decoding reports a 15% TruthfulQA gain and a 34.7% NQ-Open gain over greedy decoding for Gemma-2-2b. Neural Path Hunter reports a 20.35% relative faithfulness gain on OpenDialKG under the FeQA question-answering-based faithfulness metric, with critic precision up to 97.5%. We will reproduce these under our models rather than borrow the numbers. We expect our system to match this ceiling's factuality to within noise while cutting delay to first audio severalfold.

The strongest competitor, and the one the proposal must beat, is fixed hold-back. Streaming proceeds as normal, but every word is held for a constant delay $d$, and the checker must clear a span before it is spoken. Any span still unverified when its turn comes is spoken with silence until cleared. It uses the same language model, risk estimator, checkers and synthesizer as our policy. For each benchmark, $d$ is tuned on the same development split with the same grid budget used for our threshold. We found no reported result on Lacuna for any combined verify-while-speaking system of this kind, so this baseline has no published number. We will establish it ourselves.

Beating fixed hold-back means one of two things. Our curve of spoken-hallucination rate against delay to first audio must lie strictly below its curve over a meaningful range. Or, at matched spoken-hallucination rate, our system must reduce delay to first audio or total silent-stall time by an amount that listeners detect in the perception study.

A hard-abort variant stops audio and restarts the sentence whenever a spoken claim is refuted, which isolates the value of spliced repair.

Two ablations isolate the key ingredients. Replacing the risk estimator with uniform risk turns our policy into a checker-latency-only scheduler and tests whether knowing where fragile words are matters. Replacing fillers with plain silence at identical timing separates the effect of when we wait from how the wait sounds.

Why not just…?

Why not just hold every word back a fixed amount and verify inside that window? This is the real threat. The inequality above says it will succeed whenever the checker is fast relative to speech. Monitoring Decoding's roughly 9 tokens per second already outpaces speaking rate once speech is underway. Our bet rests on two points. The binding constraint is early claims, and slow retrieval checkers make the worst-case $d$ large. Experiment one tests this directly by varying checker latency. If fixed hold-back matches the adaptive policy at every checker speed, we will report that the speaking rate alone supplies enough slack. That is a useful negative result, and it would simplify how spoken assistants should be built.

Why not just verify the whole answer and then use ordinary batch speech synthesis? Batch synthesis latency grows with sentence length, as Ma et al. show. Whole-answer checking adds the full generation and correction time before any sound; fine-grained self-consistency runs at 316.72 ms/token in the Monitoring Decoding comparison. This baseline is included as the quality ceiling. The experiment reports how many seconds of silence it costs per turn, not merely asserts that the cost is large.

Why not just stop the audio and start over when a mistake is caught? Wester et al. found users favoured smooth reactive re-planning over non-reactive behavior. Restarting also discards correct material that preceded the error, even though Monitoring Decoding suggests errors are localized to a small fraction of tokens. The hard-abort baseline and the listener study measure whether spliced repairs such as "directed by James, sorry, Jay Roach" are preferred to restarts. If they are not, repair drops out and the policy reduces to hold-or-release.

Why not just train an end-to-end speech-to-speech model that is more factual? Such models hide the text stream that local checking needs. SHALLOW's finding that fluent decoder outputs can reverse meaning while scoring well on word error rate (a word error rate of 0.16 alongside a semantic error score of 60.79 for Phi-4 on a medical example) suggests fluency priors do not buy fidelity. SHALLOW studies speech recognition, not generation, so this is suggestive rather than decisive. We do not test end-to-end models here. Our modular pipeline requires no retraining, and its contribution is a measured delay-versus-factuality trade-off that any future end-to-end system would also have to face.

Proposed Experiments

The text side uses TruthfulQA generation judged by GPT-judge, where the original paper found the best model truthful on 58% of questions against 94% for humans. It also uses NQ-Open for short factual answers, and the dialogue portion of HaluEval together with OpenDialKG for knowledge-graph-grounded conversation, where the Neural Path Hunter checker applies. For context on detector quality, HaluEval reports ChatGPT hallucination detection at 72.40% in dialogue and 62.59% in question answering. This is why we also report results with an oracle checker, to separate policy quality from checker quality.

The speech side uses the LJSpeech-trained prefix-to-prefix synthesizer. All answers are synthesized under every policy, and timing is logged at word level.

The primary metric is the spoken-hallucination rate: the fraction of turns in which a false claim reaches the audio and is never corrected. Alongside it we report delay to first audio, total silent stall per turn, filler and repair counts, and naturalness ratings. A listener study then asks participants to rate responsiveness, naturalness and trust. It also measures whether trust tracks correctness, meaning whether listeners trust correct answers more than incorrect ones under each policy.

Two checks are decisive. The first sweeps checker latency, from the reference-model check up to a deliberately slowed retrieval check, and plots both policies' curves of spoken-hallucination rate against delay. The adaptive policy's advantage should appear and grow as checker latency rises and as risky spans move earlier in the answer. If no gap appears at any latency, the core claim is false.

The second check is the filler-versus-silence ablation at matched timing inside the listener study. If fillers do not reduce perceived stalling relative to silence, or if they lower trust in correct answers more than they help, then hesitation offers no benefit. The recommended design would then fall back to silent holds with spliced repair.