Repetition Suppression in Neural Text Generation

221 concepts137 papers1 proposal

Autoregressive language models generate text token by token, conditioning each new choice on the entire sequence of previously generated tokens. While this mechanism enables the generation of highly fluent text, it suffers from a fundamental pathology: the self-reinforcement loop.

When a model generates a phrase or sentence, that exact sequence enters its context window. Because language models are trained to detect and continue patterns, the presence of this sequence in the immediate history artificially inflates the probability of generating it again. As Xu et al. demonstrate, this creates a feedback loop where the likelihood of repeating a sequence grows monotonically with each occurrence, trapping the model in an infinite verbatim loop. Zhu et al. note that this degenerate behavior is especially severe during maximization-based decoding (such as greedy search or beam search), where the highest-probability token is consistently selected.

To break these loops, researchers have developed two primary paradigms: inference-time logit manipulation and training-time objective modification.

Inference-Time Logit Manipulation

The most direct way to prevent a model from repeating itself is to alter its token selection probabilities during decoding. This is achieved by applying a penalty to the logits (the unnormalized log-probabilities) of tokens that have already appeared.

Blunt vs. Localized Penalties

Early approaches applied a uniform frequency or presence penalty across the entire generation history. Dou et al. show that penalizing previously generated tokens improves output diversity. However, global penalties act as a blunt instrument: they punish the natural recurrence of necessary functional words (like "the", "and", or "is") and common nouns, which degrades long-form fluency.

To preserve natural language distributions, Xi et al. restrict repetition penalties to a narrow, sliding window of recent tokens (e.g., 200 tokens). This suppresses immediate local stuttering while allowing words to reappear naturally across longer narrative arcs. For absolute prevention of exact phrases, See et al. utilize n-gram blocking, a hard constraint that sets the probability of any token to zero if it would complete a previously generated sequence of length $n$.

Suffix-Aware Logit Adjustment

Even sliding-window and n-gram penalties can inadvertently destroy structural formatting. For instance, a model generating a markdown list, code block, or a dialogue script must repeat structural tokens (like -, \n, or speaker names). Standard penalties treat these structural repetitions identically to linguistic loops, breaking the intended format.

Standard Penalty:
Context: "User: Hello\nBot: Hi\nUser:" -> Penalizes "User" and ":" globally.

Suffix-Aware (DRY) Penalty:
Context: "User: Hello\nBot: Hi\nUser:" -> Only penalizes tokens that followed "User:" previously.

To solve this, the authors of the DRY (Don’t Repeat Yourself) framework introduced suffix-aware logit adjustment. Instead of penalizing a token simply because it exists in the history, the system looks at the immediate context (the suffix) preceding the current step. It identifies where that exact suffix occurred previously in the text and only penalizes the specific tokens that followed it. This allows the model to reuse individual words or structural markers freely, but surgically suppresses the exact multi-token continuations that cause verbatim looping.

Training-Time Objectives

While decoding-time interventions are computationally efficient and training-free, they do not fix the underlying model weights; they merely mask a structural flaw. To address the self-reinforcement effect at its root, researchers modify the training objective so models learn to avoid loops natively.

Unlikelihood and Negative Training

Standard Maximum Likelihood Estimation (MLE) training only teaches a model what to say, optimizing it to match a target text. It provides no explicit signal for what not to say.

The authors of Implicit Unlikelihood Training address this by combining token-level unlikelihood losses with sequence-level policy gradients. When a model generates a repetitive token during training, an unlikelihood loss explicitly minimizes its probability, forcing the model to distribute probability mass across alternative paths. Similarly, Xu et al. introduce DITTO, a fine-tuning method that exposes the model to pseudo-repetitive data and penalizes tokens that contribute to consecutive sentence-level repetitions. This teaches the model to actively "break" out of loops when it encounters repetitive context.

Reinforcement Learning and Novelty Rewards

In downstream tasks like abstractive summarization or audio captioning, models often exploit similarity-based reward functions (like ROUGE or embedding alignment scores) by repeating high-confidence keywords over and over to artificially inflate their scores.

Standard RL: Reward = Semantic Similarity(Generated, Reference) 
             (Exploitable by repeating high-confidence keywords)

Modified RL: Reward = Semantic Similarity - Repetition Penalty
             (Forces structural diversity and natural phrasing)

To counteract this gaming of the reward system, Kryscinski et al. incorporate an explicit novelty reward into reinforcement learning frameworks, penalizing the model based on the proportion of generated n-grams that copy the source text verbatim. In cross-modal generation, Ahn et al. integrate a repetition penalty directly into the reward function. This forces the model to balance semantic alignment with grammatical and structural diversity, ensuring that the model achieves high informational density without falling back on redundant, circular phrasing.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_repetition_penalty_5cbb5156,
  title = {repetition penalty},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/repetition-penalty/rdr_5cbb51567e02447dbc30af062565956b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_repetition_penalty_5cbb5156_v1,
  title = {repetition penalty},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/repetition-penalty/rdr_5cbb51567e02447dbc30af062565956b/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake