Source-Predicted Word Choices as Moves in Whole-Document Translation Search

When a machine translation system renders a document, a source term that appears ten times should usually be translated the same way each time, but not always. Today's systems handle this in one of two crude ways. They either hope a long-context neural model gets it right on its own, or they repair the output afterwards by forcing every occurrence to the same word, which damages texts whose authors vary their wording on purpose. This proposal brings back an older idea, global lexical selection: a model that reads the whole source and predicts which target words the translation should contain, independent of word order. We extend that predictor from a single sentence to the whole document. We also train it to say, for each repeated source term, whether the human translation keeps one rendering or varies it, and which renderings it uses.

We then use this predictor to drive the search, not to score finished outputs. A complete-state decoder is a decoder that holds a full translation of the document and repeatedly edits it, keeping edits that raise a score. In ours, every edit rewrites all occurrences of one repeated term at once, with the replacement sampled from the document-level predictor, and a document-level neural translation model then decides whether to keep the edit. The core claim is that this combination reaches consistent translations wherever references are consistent, preserves variation wherever references vary, and finds renderings that neither the initial translation nor a lattice built from it contains. The obvious alternatives cannot do all three.

Core Claim and What Is New

Global lexical selection was defined by Statistical Machine Translation through Global Lexical Selection and Sentence Reconstruction, which predicts a bag of target words from the source sentence with independent classifiers. It was later turned into the Discriminative Word Lexicon of Extending Statistical Machine Translation with Discriminative and Trigger-Based Lexicon Models, a log-linear decoder feature worth 0.6 to 1.0 BLEU on NIST and GALE. It was made nonlinear by the multilayer perceptron of Going Beyond Word Cooccurrences in Global Lexical Selection (+1.40 BLEU out of domain) and given syntax and embeddings in Source discriminative word lexicon for translation disambiguation.

These papers supply our method, but all of them condition only on one sentence. Mauser et al. name document context as an open extension. All of them also use the lexicon passively, as a feature or reranker, so beam pruning can discard the words the lexicon wanted before it ever scores them.

The independent-classifier design also creates the "synonym problem" that Bangalore et al. describe: valid synonyms split probability mass and each falls below threshold. Our predictor avoids this by changing what it predicts. Instead of independent yes/no decisions over the whole vocabulary, it predicts, for one repeated source term (a lexical chain), a distribution over its candidate renderings plus a binary "consistent or varied" decision. Synonyms then compete inside a single distribution rather than each failing its own threshold.

On the decoding side, Docent established document-wide local search with random phrase-level edits. Its authors observed that optimizing surface document features without a semantic lexicon caused omissions and grammatical damage. Lehrer added a chain-level "make all occurrences agree" edit and an embedding-similarity score. However, it chooses the agreed word from target-side similarity and the phrase table, never from a source-conditioned prediction, and its gains appeared in human preference (60%) but not in BLEU.

Lexical Translation Inconsistency-Aware Document-Level Translation Repair (DocRepair) is the closest neural work. It finds inconsistently translated chains, builds a lattice from the variants already present in the output, and lets a document-level Transformer choose. Our proposal differs from DocRepair in three concrete ways. Candidate renderings come from a source-document predictor, so they can lie outside the initial output. Consistency is a prediction, not an assumption, which addresses DocRepair's own stated weakness on deliberate variation. And the edit is accepted by search against a full-document translation score rather than by a single repair pass.

Research Program

The first component is the document-conditioned lexicon $q_\phi$. We identify chains by lemmatizing the source document and grouping repeated content words. For each chain we read out the aligned target renderings in the reference, obtained with word alignment, and record two things: whether the reference uses a single rendering, and which renderings it uses.

The predictor is a Transformer encoder over the whole source document that pools the representations of all occurrences of the chain. It outputs a probability that the chain is rendered consistently, a distribution over renderings for the consistent case, and per-occurrence distributions for the varied case. Candidate renderings come from a phrase table or alignment dictionary built on the training corpus, so the inventory is not limited to what the baseline system happened to produce. We train on the Parallel Document Corpus introduced in Rethinking Document-level Neural Machine Translation (about 60k Chinese-English documents, 1.39M sentences). We then evaluate the predictor directly: rendering accuracy, and calibration of the consistency decision, compared against a sentence-only version of the same predictor. This isolates whether document context helps lexical choice at all, which is the claim Mauser et al. left open.

The second component is the decoder. It starts from the output of a strong document-level neural model and maintains the full document translation. Each step picks a chain, samples a rendering pattern from $q_\phi$, and substitutes the new renderings at the aligned target spans. It then re-decodes only the affected sentences with the new words imposed as hard lexical constraints, so that agreement and word order around the substituted words are repaired. The resulting document $D$ is scored as

$$S(D) = \log p_\theta(D \mid X) + \lambda \sum_{c} \log q_\phi\big(r_c(D) \mid X, c\big),$$

where $X$ is the source document, $p_\theta$ is the document-level translation model's probability of $D$, $r_c(D)$ is the rendering pattern that $D$ uses for chain $c$, and $\lambda$ is tuned on development data. Edits are accepted under simulated annealing. Following Docent's two-step scoring, a proposal is rejected without full rescoring when its lexicon-score change alone cannot clear the acceptance threshold.

Docent-style decoders are known to give different outputs for different random seeds. We address this by running a fixed number of chains and keeping the highest-scoring final state, and we report variance across seeds.

The third component is analysis. We split test chains by whether the reference renders them consistently or varies them, and report consistency and accuracy separately on each group. This is the test that "always enforce consistency" methods cannot pass.

Baselines

The naive reference point is sentence-by-sentence translation with a standard Transformer, plus that same model applied to concatenated documents. These show how much consistency comes for free.

The strongest pure-generation competitor is the multi-resolution document-to-document Transformer of Rethinking Document-level Neural Machine Translation. It is trained on mixed sentence and full-document lengths with tuned dropout, and reports 25.9 d-BLEU on TED Chinese-English against 19.1 for a specialized architecture. It matters because a model that already attends over the whole document might make our search redundant. It also serves as our initial translation and as $p_\theta$, so any gain from our method is measured over exactly this model. We found no reported consistency (LTCR) result for it on Lacuna, so we will measure it ourselves.

The strongest consistency method is DocRepair. On NIST Chinese-English it reports BLEU rising from 48.45 to 50.28, LTCR from 65.25 to 69.51, and ref-LTCR from 78.61 to 80.74. We will apply DocRepair to the same multi-resolution output, train its repair model on the same corpus, and give it the same development-set tuning budget. Beating it means higher ref-LTCR, which rewards matching the reference's consistency pattern, at equal or better BLEU on NIST and TED. It also means a clear advantage on the reference-varied subset, where DocRepair's consistency assumption should hurt.

Two baselines test whether the search itself matters. The first is $N$-best reranking: rerank document $N$-best combinations with the same $q_\phi$ score, following the greedy document reranking of Improve SMT with Source-Side "Topic-Document" Distributions. The second runs the same search with Lehrer-style proposals, where every chain is forced to its majority rendering in the current output.

Two ablations isolate the key ingredients: a sentence-conditioned $q_\phi$, and a $q_\phi$ whose consistency decision is fixed to "always consistent". All systems get matched decoding compute, measured in translation-model forward passes per document.

Why not just…?

Why not just trust the document-level Transformer? It sees the whole document, and its strong d-BLEU shows that it uses context. But BLEU is largely blind to consistency: Lehrer's human-preferred gains left BLEU flat. Separately, A Test Suite for Evaluating Discourse Phenomena in Document-level Neural Machine Translation finds that context-aware models fall back on frequent defaults rather than using context. This is a real threat. If the multi-resolution model's ref-LTCR already equals the reference-pattern ceiling our predictor reaches, the proposal fails, and the first experiment measures exactly that.

Why not just run DocRepair? DocRepair can only choose among variants present in the initial output, and it assumes consistency is always desired. Both limits are measurable. We will count how often the reference rendering is absent from the initial output, which sets the ceiling on what lattice repair can recover. We will also compare against DocRepair on the subset where references vary deliberately.

Why not just add a document-level word lexicon as a reranking feature? Gong et al. show that rerankers cannot recover candidates that the decoder pruned. The reranking baseline above uses the identical $q_\phi$, so any gap between it and our search measures the value of using the predictor to propose edits, independent of the predictor's quality.

Proposed Experiments and Decisive Checks

We train on PDC and test on NIST Chinese-English and TED Chinese-English. We report BLEU and d-BLEU, LTCR, and ref-LTCR overall and on the consistent and varied subsets. We also run the discourse test suite as a check that the edits do not damage pronouns or connectives, and a document-level human preference study on NIST.

Three results decide the proposal. First, the document-conditioned predictor must beat the sentence-conditioned one on rendering accuracy for chains whose disambiguating cues lie in other sentences. Second, full search must beat reranking with the same predictor on ref-LTCR at equal compute. Third, on the reference-varied subset, the learned consistency decision must beat both the "always consistent" ablation and DocRepair without losing BLEU on the consistent subset. If the first check fails, document context does not help lexical choice. If the second fails, the search adds nothing over reranking. If the third fails, the consistency-versus-variation predictor is not earning its place.