Speech recognizers that map audio straight to text in one network still get rare names wrong. They swap an unfamiliar executive's surname or a niche product for a common word that sounds similar. The standard fix is contextual biasing: give the recognizer a list of words likely to appear, mined from slides or press releases, and boost them during decoding. On real meetings this fix runs into two opposing problems. The mined lists are incomplete. In earnings calls they contain 82% of spoken person names but only 39% of spoken product names. Yet adding more candidates makes the recognizer "over-bias", inserting listed names the speaker never said. This note proposes treating the choice of biasing list as a next-item recommendation problem. Before each stretch of audio, a small model reads what has been recognized so far. It then looks at a graph of entities connected through shared documents and co-mentions, and predicts which entities are likely to be spoken next, including some that never appeared in the mined documents. Only that short, refreshed list is handed to an existing biasing module.
The idea borrows one lesson from recommender systems that combine interaction sequences with graphs: the recent sequence decides which edges of a static graph matter right now. Here the graph does what the sequence cannot. It proposes entities absent from the context documents. The conversation does what the graph cannot. It throws away most of those proposals before they can cause false insertions. The central claim is that these two parts interact: graph expansion alone should hurt, sequence-only scheduling cannot recover unlisted entities, and only the combination should lower rare-word error on the entities the documents missed.
Core Claim
On long-form, real-context speech recognition, a biasing list that is (a) expanded through an entity graph and (b) re-ranked every segment from the running transcript will reduce rare-word error below both the static document-mined list and a sequence-only re-ranker. The gain should be concentrated on entities missing from the mined documents. It should come without the rise in false insertions that naive expansion causes. The test is an interaction effect in a two-by-two design (graph expansion on or off, sequential filtering on or off), not a single headline number.
Where This Sits
The motivating evidence comes from ConEC. It builds an earnings-call benchmark with context mined from real slides and documents. It shows that a Zipformer transducer (a streaming recognizer that emits text frame by frame) improves rare-word error only from 26.02% to 24.84% with this context, while an oracle use of the same context would reach 18.72%. It also shows that Whisper Large, despite 680,000 hours of training audio, errs on 28.9% of person names. That gap, and the 39% product coverage, are the targets.
Two prior papers sit closest to this intersection. Tree-constrained Pointer Generator with GNN encodings runs a graph network over a prefix tree of word pieces. Each partial spelling can then "see" which complete words it may become, which lowers out-of-vocabulary error from 73% to 33% on LibriSpeech. Its graph is spelling only: it has no relations between entities, and the list is a static, unordered bag. We keep that kind of biaser as a downstream component and change what goes into it. Using Knowledge Graph and Search Query Click Logs does use entity relations and dialog history. An entity recognized in one turn boosts knowledge-graph neighbors in the next turn's n-best list, which cut WER from 16.2% to 15.1% in a voice assistant. But it works as second-pass rescoring inside an n-gram system with hand-set shortest-path scores, so an entity pruned before the n-best list can never be recovered. We move the relational step before decoding, where it decides what the end-to-end biaser may attend to, and we learn the scoring instead of hand-setting it.
The scaling and retrieval literature constrains the design. Retrieve and Copy prunes catalogs of up to 20,000 items to the 10 nearest by embedding before attention. It reports a 6% WER reduction and 20% faster decoding, but also regressions on long VoxPopuli audio from over-biasing. That is evidence that list size per segment, not total catalog size, is what matters on long-form audio. CTC-Assisted LLM-Based Contextual ASR filters a 2,000-word list by edit distance against a rough first-pass transcript before prompting an LLM decoder. Like Retrieve and Copy, it can only select from the given list. Neither can add an entity the documents missed.
The method itself comes from sequential-graph fusion. Integrating Sequential and Relational Modeling for User Events finds that a two-stage design works well: a sequence encoder produces embeddings that are fed as features to a graph network, raising MRR on Brightkite from 40.2 to 47.4. It also finds that turning every sequential event into a graph node degrades accuracy and that fully joint temporal graph networks run out of memory. We therefore adopt the two-stage pattern and do not make each utterance a node. Graph-Sequential Alignment and Uniformity aligns a sequence embedding with graph item embeddings on a hypersphere and reports 5.16–13.81% gains over SASRec and LightGCN. It also reports that the gain depends on tuning the loss weight near 0.1. We use its alignment loss as an optional ingredient, ablate it, and treat its sensitivity as a known risk. The broader framing of short-term sequence against long-term graph comes from work such as STP-UDGAT and news recommendation with long- and short-term interests. In those papers the sequence exploits known history while the graph explores unvisited but related nodes, which is the role split we use.
Research Program
The first piece is the entity graph. Nodes are named entities (persons, organizations, products) extracted from the ConEC context documents of every call in the training companies, plus the companies themselves. Edges link entities that co-occur in a document, entities mentioned in the same call transcript, and an entity to its company and to companies in the same call series or sector where the metadata provides it. For a test call, the candidate pool is that call's mined entities plus their one- and two-hop neighbors. Graph edges come only from training-fold companies and from the test call's own documents, never from its transcript.
The second piece is the next-entity predictor, which follows the two-stage pattern. A text encoder reads the recognizer's own output for the preceding segments, so the predictor sees recognition errors at test time exactly as it would in deployment. A graph network produces an embedding for each candidate entity. The predictor scores each candidate for the upcoming segment $t$ as
$$s(e \mid t) = \langle q(h_{<t}), g(e) \rangle + \beta \, m(e),$$
where $h_{<t}$ is the hypothesis text before segment $t$, $q$ is the sequence encoder's projection, $g(e)$ is the graph embedding of entity $e$, and $m(e)$ is a binary indicator that $e$ was in the call's mined documents, weighted by a learned scalar $\beta$. The indicator lets the model trust listed entities more than graph-proposed ones. The top $k$ entities by $s$ become segment $t$'s biasing list. The predictor is trained on text alone: from training transcripts we build examples asking which entities are spoken in segment $t$ given segments before it, using a sampled-softmax loss with the other pool members as negatives. Following Contextual Speech Recognition with Difficult Negative Training Examples, we oversample negatives that sound like the positive, so the predictor learns not to promote acoustic look-alikes.
The third piece is the recognizer, which we leave unchanged. We use the ConEC Zipformer transducer with its icefall biasing so our numbers sit directly next to the published 24.84%. As a second, neural biaser we use the tree-constrained pointer generator from Minimising Biasing Word Errors, with its internal-LM discounting. This tests whether the gain holds across biasing mechanisms. Because the recognizer is frozen, any improvement is attributable to list construction.
Baselines
The naive reference point is the ConEC Zipformer transducer with no biasing, which ConEC reports at 26.02% rare-word WER. Whisper Large is reported as a second reference using ConEC's entity-type error rates: 28.9% on PERSON and 19.6% on ORG.
The strongest existing method on this exact benchmark is the static full mined list used by ConEC, at 24.84% rare-word WER, with the oracle at 18.72% and synthesized context at 23.43%. We rerun it in our pipeline with the same model and decoding settings and require our reproduction to land near 24.84% before comparing. Its biasing weight gets the same search budget as our $k$ and $\beta$, chosen on held-out companies. Beating it means lower rare-word WER at equal or lower false-insertion rate on biased entities, with the gap larger than the variation across company-level cross-validation folds. We also report how much of the 24.84-to-18.72 gap we close.
The strongest retrieval competitor is a per-segment acoustic pruning baseline in the spirit of Retrieve and Copy and CTC-Assisted LLM-Based Contextual ASR. It matches a first-pass hypothesis for the segment against the mined list by phonetic edit distance and keeps the top $k$ entries, with the same $k$ as ours. No reported result was found on Lacuna for either method on ConEC, so we run both ourselves. We also give acoustic pruning access to the graph-expanded pool, so the graph's benefit is not credited only to our method.
The key ablations form the two-by-two design. Sequence-only uses the same predictor with candidates restricted to the mined list. Graph-only uses the expanded pool ranked by graph proximity to the call's mined entities, fixed for the whole call. A recency heuristic biases toward mined entities plus any entity already recognized. A further ablation adds or removes the alignment loss of Graph-Sequential Alignment and Uniformity. All variants share the recognizer, $k$ search, tuning budget and folds.
Why not just…?
Why not just use a bigger foundation model? ConEC already tests this: Whisper Large, trained on 680,000 hours, still misses 28.9% of person names in these calls. Company-specific product names are too rare in any audio corpus to be memorized. We report Whisper as a reference row but do not expect scale to close the gap.
Why not just bias with the whole mined list and tune the weight? That is the 24.84% baseline. Its ceiling is set by coverage: an entity absent from the list, which is 61% of spoken products, cannot be boosted at any weight. Raising the weight on a longer list is exactly what produced the over-biasing regressions reported in Retrieve and Copy. Our per-type results on unlisted PRODUCT entities directly test whether coverage, not weighting, is the bottleneck.
Why not just prune by acoustic match each segment? Acoustic pruning is a strong way to choose within a list, and it may match us on listed entities. It cannot propose unlisted ones, and it is weakest where the first pass most corrupts an unfamiliar name. The variant that gives acoustic pruning the graph-expanded pool settles whether the conversation signal adds anything beyond expansion plus phonetic matching. If that variant ties our method, the honest conclusion is that the graph matters and the sequence encoder does not, and we will report it that way.
Why not just have an LLM read the transcript so far and the documents, and guess the upcoming entities? This is the most serious threat, because a sequence-only model may capture most of the scheduling benefit. That is why the claim was narrowed to the interaction: a text model reading only the documents can name only what the documents contain, or invent names. The sequence-only ablation, and a prompted LLM variant given the same history and documents, will show whether graph-proposed candidates add recall on unlisted entities. If the LLM achieves comparable unlisted-entity recall without the graph, the graph component is unnecessary and the proposal reduces to sequential scheduling.
Why not just put the conversation and entities into one graph? Integrating Sequential and Relational Modeling for User Events found that turning sequential events into graph nodes lowered accuracy and that joint temporal graph networks ran out of memory. We follow its two-stage result and do not retest that design.
Proposed Experiments
The first experiment is a cheap check that can stop the project. Before any training, we measure on ConEC how many spoken entities missing from a call's mined documents appear in its one- and two-hop graph neighborhoods, and how large those neighborhoods are. If two-hop expansion recovers only a small share of missing products, or does so only at pool sizes in the thousands, the expansion premise fails and the project reduces to sequential scheduling. We will report whichever outcome holds.
The second experiment is offline next-entity prediction. Its metric is recall@$k$ of entities actually spoken in the next segment, reported separately for listed and unlisted entities, across the four ablation cells and the LLM variant. This isolates ranking quality from recognizer behavior.
The main experiment is end-to-end recognition on ConEC with both biasers. We report overall WER, rare-word WER, error per entity type split by listed versus unlisted, and the false-insertion rate of biased entities. The decisive pattern is that graph-only raises false insertions and does not lower rare-word WER, sequence-only lowers it only on listed entities, and the full model lowers it on unlisted PRODUCT and ORG entities without raising false insertions. We sweep $k$ from 5 to 100. If the advantage disappears at large $k$, that indicates per-segment list size is the operative factor.
The final experiment tests whether the result transfers beyond earnings calls. It uses AMI meetings with slide-OCR biasing lists as in the GNN tree-constrained pointer generator work, for which no reported number was found on Lacuna. Here the graph is built only from slide text, so success would show the method does not depend on cross-company metadata. LibriSpeech is used only to confirm that the frozen biasers are reproduced correctly. Its utterances carry no conversation history, so it cannot test the claim.
References
- ConEC: Earnings Call Dataset with Real-world Contexts for Benchmarking Contextual Speech Recognition
- Tree-constrained Pointer Generator with Graph Neural Network Encodings for Contextual Speech Recognition
- Using Knowledge Graph and Search Query Click Logs in Statistical Language Model for Speech Recognition
- Retrieve and Copy: Scaling ASR Personalization to Large Catalogs
- CTC-Assisted LLM-Based Contextual ASR
- Integrating Sequential and Relational Modeling for User Events: Datasets and Prediction Tasks
- Graph-Sequential Alignment and Uniformity: Toward Enhanced Recommendation Systems
- STP-UDGAT: Spatial-Temporal-Preference User Dimensional Graph Attention Network for Next POI Recommendation
- Graph neural news recommendation with long-term and short-term interest modeling
- Contextual Speech Recognition with Difficult Negative Training Examples
- Minimising Biasing Word Errors for Contextual ASR With the Tree-Constrained Pointer Generator