Flexible Alignment Objectives for Sequence Generation

42 concepts27 papers1 proposal

The Tyranny of the Rigid Grid

Standard sequence generation models are traditionally trained using token-wise Cross-Entropy (CE) loss. This objective imposes a strict, index-by-index matching criterion between the predicted distribution and the ground-truth reference sequence. It treats the target sequence as a rigid grid where token $A$ must appear exactly at position $t$.

To understand why this is problematic, consider a model tasked with generating a sentence. If the ground truth is The cat sat on the mat and the model predicts [blank] The cat sat on the mat, a token-wise comparison will compare The against [blank], cat against The, sat against cat, and so on.

Because of a single leading insertion or shift, every single subsequent token is marked as incorrect and heavily penalized. As noted by Marjan Ghazvininejad et al. in Aligned Cross Entropy for Non-Autoregressive Machine Translation, this rigid alignment constraint introduces severe noise into the training signal. It forces models to prioritize exact slot-matching over semantic or grammatical correctness, penalizing valid sequence variations and minor positional offsets.

The Parallel Generation Bottleneck

This positional rigidity becomes a catastrophic bottleneck in non-autoregressive generation. While traditional autoregressive models generate text step-by-step—allowing them to adapt to their own structural shifts—non-autoregressive translation (NAT) models predict all tokens simultaneously in parallel to accelerate inference, as surveyed by Yisheng Xiao et al. in A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond.

When a source sentence can be translated into multiple valid target word orders—a challenge known as syntactic multi-modality (Kexun Zhang et al., A Study of Syntactic Multi-Modality in Non-Autoregressive Machine Translation)—a rigid loss function penalizes the model for choosing any path other than the exact token positions of a single reference.

Faced with conflicting training signals, parallel models hedge their bets by predicting multiple valid words for the same position, resulting in severe token repetition, stuttering, and incoherent outputs (Jiatao Gu et al., Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade). To resolve this, researchers have shifted toward flexible alignment objectives that decouple token correctness from fixed positions.

Monotonic Latent Alignment via Dynamic Programming

The first major family of flexible losses relaxes the index-to-index constraint while preserving the chronological order of tokens. Instead of forcing a rigid $t$-to-$t$ mapping, these objectives find the optimal monotonic path between predictions and targets using dynamic programming.

In Aligned Cross Entropy for Non-Autoregressive Machine Translation, Marjan Ghazvininejad et al. introduced Aligned Cross Entropy (AXE). AXE constructs a cost matrix between all predicted distributions and target tokens, then uses a dynamic programming formulation to find the lowest-cost monotonic alignment. If the model shifts a phrase by two positions, the dynamic programming path simply takes two horizontal or vertical steps (representing skips or insertions) to realign with the sequence. The model is penalized only for the structural edit rather than receiving a blanket penalty for every subsequent token.

Rigid CE Alignment:          AXE Monotonic Alignment:
Target:    A  B  C  D        Target:    A  B  C  D
           |  |  |  |                   \  \  \  \
Predicted: _  A  B  C        Predicted: _  A  B  C  D
(Every token penalized)      (Zero penalty after alignment)

This intuition generalizes across domains where sequence lengths vary or noise shifts tokens:

Going Order-Agnostic with Bags and Permutations

While monotonic alignments absorb local shifts, they fail when language pairs require global reordering (e.g., translating from a Subject-Verb-Object language to a Subject-Object-Verb language). To handle this, a second family of objectives removes positional constraints entirely during parts of training.

An early manifestation of this idea was proposed by Shuming Ma et al. in Bag-of-Words as Target for Neural Machine Translation, where a standard sequence loss is supplemented with a Bag-of-Words (BoW) target. This rewards the model for predicting the correct lexical content anywhere in the sentence, mitigating the rigid penalization of valid alternative phrasings.

To make this concept fully operational for parallel token generation, Cunxiao Du et al. introduced Order-Agnostic Cross Entropy (OAXE) in Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation. Instead of finding a monotonic path, OAXE uses the Hungarian matching algorithm to find the absolute best permutation of the target sequence that minimizes cross-entropy against the model's parallel predictions. This allows the model to focus entirely on learning what words to generate during early training phases before being forced to master where they belong.

Balancing Global Flexibility and Local Syntax

The primary drawback of pure order-agnostic training is that it can destroy local syntax, as a completely scrambled sentence achieves the same loss as a grammatically perfect one. To reintroduce syntactic structure without sacrificing positional flexibility, researchers extended order-agnostic losses to phrases and $n$-grams:

  1. Bag-of-Ngrams Difference: Chenze Shao et al. proposed an objective in Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translation that minimizes the discrepancy between predicted and target $n$-gram distributions globally, ensuring local phrases are preserved even if their global positions shift.
  2. Phrase-Based OAXE (ngram-OAXE): In ngram-OAXE: Phrase-Based Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation, Cunxiao Du et al. modified OAXE to treat contiguous $n$-grams as the fundamental units of alignment. This enforces local word order within phrases while allowing those phrases to be reordered globally.
  3. Edit-Invariant Sequence Loss (EISL): Guangyi Liu et al. generalized this in Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generation by calculating the matching probability of target $n$-grams across all possible positions in a generated sequence, making text generation robust to noisy or paraphrased training data.

Non-Monotonic and Fuzzy Latent Alignments

Between strict monotonicity and complete order-agnosticism lies the realm of non-monotonic latent alignments, which explicitly model how words cross paths during translation.

In Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translation, Chenze Shao et al. introduced NMLA, an objective that allows latent connections between input and output sequences to cross. By maximizing $n$-gram matching scores over non-monotonic paths, the model learns global reordering patterns natively within a Connectionist Temporal Classification (CTC) framework.

For graph-based architectures like Directed Acyclic Transformers, Zhengrui Ma et al. developed Fuzzy Alignments (FA-DAT) in Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translation. Instead of enforcing a strict token-to-node match, FA-DAT evaluates the expected output of a graph against reference $n$-grams. This integrates a soft, BLEU-like overlap metric directly into the differentiable optimization process, allowing the model to explore diverse linguistic structures without positional penalties.

Why Latent Alignments Fail in Autoregressive Models

Given the success of flexible alignment losses in parallel generation, a natural question arises: Can we use them to improve standard autoregressive models like GPT?

Adi Haviv et al. investigated this question in Can Latent Alignments Improve Autoregressive Machine Translation? and uncovered a fundamental incompatibility with teacher forcing.

When an autoregressive model is trained using teacher forcing, it receives the ground-truth prefix at every step. If a flexible alignment loss (like AXE or CTC) is applied in this setting, the model discovers a trivial shortcut: it can achieve near-zero training loss by simply copying tokens directly from its immediate context history rather than learning the actual translation mapping. Because the loss function allows for positional shifts, the model is never penalized for this copying behavior during training.

Consequently, when teacher forcing is removed at inference time, the model's generation completely collapses. Flexible alignment objectives remain a specialized, powerful tool uniquely suited for parallel, non-autoregressive generation where tokens are predicted independently and cannot be copied from a sequential history.

Key Foundational Work

Edit Probability for Scene Text Recognition (2018) — contributed 1 concept elements to this direction.
Effectively processing medical term queries on the UMLS Metathesaurus by layered dynamic programming (2014) — contributed 1 concept elements to this direction.
Bag-of-Words as Target for Neural Machine Translation (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_rigid_sequence_alignment_29dcf392,
  title = {rigid sequence alignment},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/rigid-sequence-alignment/rdr_29dcf39299dc420eaa12c63ac5853a5c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_rigid_sequence_alignment_29dcf392_v1,
  title = {rigid sequence alignment},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/rigid-sequence-alignment/rdr_29dcf39299dc420eaa12c63ac5853a5c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake