Breathing as a Commit Signal for Streaming Retrieval in Spoken Assistants

Voice assistants that look things up while you are still talking face a hard choice whenever you pause. If the pause ends your question, the system should already be searching and answering. If you are only gathering your thoughts, a search launched now uses half a question, retrieves the wrong evidence, and may lead the assistant to talk over you. Microphones have little to work with in that moment, because a silent pause sounds the same either way. Your chest does not. People who plan to keep talking take a short, quick breath soon after they stop. People who are done tend to keep breathing out toward their resting level. Respiratory inductance plethysmography (RIP), which uses elastic bands with wire coils around the rib cage and abdomen whose inductance tracks lung volume, records this silently and continuously.

We propose to use the listener-facing user's own RIP signal as the gate that decides when a streaming retrieval system commits: when it treats the question as complete, fixes its search query, and starts answering. The claim is narrow and testable. Speculative retrieval can stay cheap and acoustic, but the commit decision during silent pauses should be conditioned on breathing. At matched first-token latency, this should cut answers to incomplete questions, wasted retrieval calls, and takeovers during thinking pauses, compared with the best audio-only controllers.

Core Claim and Positioning

Three findings motivate this. Respiratory studies show that inhalations made before continuing a turn are shorter and start sooner after the speaker stops than other breaths. Respiratory Turn-Taking Cues found that inhalation duration and delay predict turn-keeping in three-party Swedish conversation, measuring amplitude against a resting expiratory level (REL) estimated causally over the previous 20 breathing cycles. Improving Prediction of Speech Activity Using Multi-Participant Respiratory State showed that the slope of lung volume over 100 ms frames predicts a person's own upcoming speech better than lung volume itself. It also found that one's breathing predicts one's own speech but not the partner's. That last result matters for the design. It means the bands belong on the user, and the signal answers exactly the question "is the user about to say more?"

On the retrieval side, Stream RAG issues tool calls while the user is still speaking, either by polling every second or through a model trained to emit calls when context seems sufficient. It reaches negative first-token latency on spoken question answering. Its triggers are purely acoustic, and the file notes that early triggering on incomplete context causes redundant compute and irrelevant evidence. Full-Duplex-Bench documents the matching turn-taking failure: fast models such as Moshi take over the floor during user pauses.

No work in the file feeds physiological signals into a retrieval controller. The closest work therefore splits in two. The RIP studies predict speech activity but never act on the prediction. Stream RAG acts on its predictions but cannot sense silent pre-speech breathing.

Two papers constrain the design. First, the respiratory effect sizes are modest. The R² values in Respiratory Turn-Taking Cues run from 0.03 to 0.16, and the test-set gain in the 2017 study is 0.006 bits per 100 ms. We do not expect breathing to replace acoustic turn models. We expect its value to sit in the small set of frames that matter most, the silent pauses, where averages over whole recordings hide it. Second, the 2017 study normalized signals with a whole-recording Z-score that looks into the future. We restrict ourselves to the causal REL tracking of the 2016 paper and to running statistics.

Research Program

The system has three parts.

The first is a speculative layer. It is a model-triggered retriever in the style of Stream RAG that may issue cheap provisional queries from partial transcripts. A Speculative RAG-style drafter, which writes candidate answers from subsets of retrieved documents for a larger model to verify, may prepare drafts from the results.

The second is a commit controller. During every 100 ms frame of user silence, it estimates the probability that the user will resume speaking within the next second. It conditions on the acoustic and transcript state, which comes from a voice-activity projection model (a model that predicts upcoming speech activity from audio). It also conditions on causal RIP features: inhalation onset since the pause began, rib-cage and abdominal slope, current level relative to the running REL, and time since last exhalation peak.

The third is a policy that maps this probability to one of three actions: commit and answer, keep speculating with a revisable query, or discard the current speculative branch. The threshold is set to trace a latency–error curve rather than to report a single point.

We will also build the data this needs, because no existing corpus pairs RIP with spoken information requests. Around 40 participants will wear calibrated dual RIP bands and a headset microphone and pose spoken questions to a live assistant. The questions are drawn from AudioCRAG and SLUE-SQA topics, but participants paraphrase them in their own words. Prompts are designed to produce multi-part and self-corrected questions, so that thinking pauses occur naturally. Every pause is annotated for whether the user resumed and whether the question was semantically complete at that point. That annotation lets us score "answered too early" directly. The live sessions use a conservative silence-threshold endpointer, so that pauses are not cut short by the system under test. All controllers are then evaluated offline by replaying the same synchronized streams.

Baselines

The naive reference point is a cascaded pipeline that waits for a fixed silence threshold, then transcribes, retrieves, and answers. It never answers too early on long pauses but is slow, and it fixes the latency ceiling we must beat.

The strongest existing retrieval competitor is Stream RAG in both its fixed-interval and model-triggered forms, rerun on our corpus with the same retriever, index, generator, and hardware. Its reported SLUE-SQA result is 55.8% exact match with −0.68 s first-token latency. On AudioCRAG it reduced latency from about 3.16 s to about 1.36 s and improved accuracy by 20.0 points absolute over closed-book models. Those numbers come from scripted or read audio, so they are not directly comparable to our spontaneous questions. We report them only as context and judge the comparison on our own replayed streams.

The strongest turn-taking competitor is voice activity projection, the top system in TurnBench with 0.845 end-of-turn recall and 0.945 interruption recall. We fine-tune it on the same training split and with the same tuning budget as our controller and plug it into the same commit policy. That makes the comparison purely about the input signal.

Two ablations isolate breathing. One feeds the controller RIP features time-shifted by several seconds, which keeps their statistics but breaks their timing with the pause. The other drops the slope features and keeps only static level. The second ablation tests the 2017 finding that dynamics carry the information.

Beating these baselines means one of two things on the held-out participants. Either the RIP-conditioned controller has lower premature-commit rate at equal median first-token latency, or it has lower latency at equal premature-commit rate. In both cases answer accuracy and retrieval calls per question must be no worse, with confidence intervals from a participant-level bootstrap.

Why not just…?

Why not just estimate breathing from the microphone instead of wearing bands? Deep Sensing of Breathing Signal During Conversational Speech recovers breathing from spectrograms with 91.2% breath-event sensitivity. However, it uses 4-second windows of voiced speech and by construction cannot see mechanics during silence, which is exactly where our decision lives. This is still a real threat, because a close headset microphone can pick up audible inhalation noise. We test it head-on: a dedicated audio-only inhalation detector, trained on the same pauses with RIP-derived labels, replaces RIP in the controller. If it matches RIP, the honest conclusion is that the bands are unnecessary, and the contribution becomes breath-aware commit control from audio.

Why not just use a better acoustic turn model such as voice activity projection? It is strong on end-of-turn recall. TurnBench reports, however, that such models fail to match the human median turn transfer of −151 ms without a rise in false positives, and Full-Duplex-Bench shows takeovers during pauses. The fusion-versus-acoustic comparison inside our baselines settles this. We pre-register that the benefit should be concentrated in pauses longer than 300 ms. If it is not, the idea fails.

Why not just learn a value-based stopping controller for retrieval, as in Stop-RAG, or let speculative drafting absorb the cost? These methods decide how much to retrieve given the information already available, and they can be made cheaper. They cannot know whether the user is about to add a clause that changes the question. We include a learned acoustic-only commit controller with the same architecture and budget. The gap between it and the RIP version is the measurement of what breathing adds beyond better control.

Proposed Experiments and Decisive Checks

We first validate the predictor offline. On the Swedish and Estonian RIP corpora used by the 2016 and 2017 studies, a strictly causal version of the pause-continuation predictor should reproduce their direction of effects. We then measure the same predictor's area under the curve on pauses in our new corpus, split by pause length.

The main experiment replays all held-out sessions through every controller. We plot first-token latency against premature-commit rate. Alongside the curve we report exact match against reference answers, retrieval calls and generated tokens per question as a compute proxy, and a Full-Duplex-Bench-style takeover rate on intra-turn pauses.

The decisive check is the pause-stratified comparison against voice activity projection plus Stream RAG. RIP must improve the curve on silent pauses. It should not matter on mid-speech frames, which serves as a sanity check against leakage. It must also survive the time-shift ablation and the headset inhalation-detector alternative.

A final test addresses generalization across people, the most likely failure mode. We train on one posture and set of participants and test on others, with and without running REL normalization. We also report how many seconds of calibration a new user needs before the gain appears.