Fixed Delay Policies for Simultaneous Machine Translation

This direction is merged into another direction: latency-quality trade-off. Its last version stays readable and citable here.

Version 1 of 140 papers43 conceptsfrozen 2026-10-04

In traditional machine translation, the model is a "perfectionist": it waits until a speaker finishes an entire sentence before attempting to translate a single word. While this ensures maximum context, it creates a lag that makes real-time conversation impossible. Simultaneous Machine Translation (SimulMT) changes the objective from "translate the sentence" to "translate the stream."

The core challenge of SimulMT is the Read-Write decision. At any given moment, should the system wait for more information (READ) or commit to an output word (WRITE)?

The Wait-k Strategy

Early attempts to solve the Read-Write problem relied on complex reinforcement learning agents. Jiatao Gu and colleagues in Learning to Translate in Real-time with Neural Machine Translation proposed an agent that learned to balance quality and latency by choosing between these actions. However, these dynamic policies were often difficult to train and unpredictable in production.

The field shifted toward a simpler, more robust approach: the wait-$k$ policy. Introduced by Mingbo Ma, Liang Huang, and their team in STACL, the wait-$k$ policy is a deterministic schedule:

  1. Wait for exactly $k$ source tokens.
  2. Alternate between writing one target token and reading one new source token.

This creates a constant lag of $k$ tokens. If $k=3$, the model is always three words behind the speaker. This predictability is powerful because it allows the attention mechanism to stabilize, as noted by Fahim Dalvi in Incremental Decoding and Training Methods for Simultaneous Translation.

Learning to Anticipate

The wait-$k$ policy works because of a framework called prefix-to-prefix training. In standard translation, the model learns to map a full source sentence to a full target sentence. In prefix-to-prefix training, the model is trained to predict the next target word using only a partial prefix of the source.

This forces the model to develop implicit anticipation. For example, when translating from German (a verb-final language) to English, the model might need to generate an English verb before the German verb has even been spoken. By training with a fixed $k$ delay, the model learns the statistical likelihood of what the speaker is about to say, effectively "hallucinating" the future context based on the partial evidence it has already received.

Scaling to Large Language Models

As the field has moved toward Large Language Models (LLMs), the wait-$k$ logic has proven surprisingly adaptable. Lizhong Chen and the Simul-LLM team demonstrated that off-the-shelf LLMs can perform simultaneous translation simply by adjusting the prompting strategy to output one word at a time.

More advanced frameworks like FASST by Siqi Ouyang and Lei Li optimize this further for speech. They use "consistency masks" and incremental encoding to ensure the LLM doesn't re-process the entire audio stream every time a new segment arrives, reducing computational overhead while maintaining the fixed-delay benefits.

Beyond the Algorithm: The Human Factor

While researchers often focus on the BLEU score (quality) vs. Average Lagging (latency) trade-off, Sara Papi and Matteo Negri argue in Visualization: The Missing Factor in Simultaneous Speech Translation that a third pillar is essential: user experience.

A low-latency system that flickers or changes its mind (re-writing words as more context arrives) creates a high cognitive load for the reader. This has led to a preference for "monotonic" policies—like wait-$k$—which never look back. Once a word is written, it is final. This constraint makes the translation task harder for the AI but significantly easier for the human reading the subtitles.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_fixed_delay_policies_for_simultaneous_ma_249fab2c,
  title = {Fixed Delay Policies for Simultaneous Machine Translation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fixed-delay-policies-for-simultaneous-machine-translation/rdr_249fab2c0e934dcfb62099b2019a2210},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_fixed_delay_policies_for_simultaneous_ma_249fab2c_v1,
  title = {Fixed Delay Policies for Simultaneous Machine Translation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fixed-delay-policies-for-simultaneous-machine-translation/rdr_249fab2c0e934dcfb62099b2019a2210/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}