BLooP: Zero-Shot Abstractive Summarization using Large Language Models with Bigram Lookahead Promotion

Varun Iyer, Cornelia Caragea

TL;DR

BLooP (Bigram Lookahead Promotion) is a simple, training-free decoding intervention that significantly improves the factual accuracy of LLM-generated summaries. By applying a small "nudge" to the probability of tokens that form bigrams found in the source text, BLooP grounds models like Llama-3.1 and Gemma-2 in the source facts without sacrificing their natural fluency.

What Problem Does This Paper Address?

Large Language Models (LLMs) are remarkably good at "zero-shot" summarization—writing a summary without being specifically trained for it. However, they suffer from two major issues: hallucinations (making things up) and omissions (missing key details).

Traditionally, researchers solved this using "copy mechanisms" that allow a model to point back to the source text. The catch? Most of these methods require expensive fine-tuning on massive datasets or require access to the model's internal attention weights, which is computationally heavy and often incompatible with optimized inference engines like vLLM. The authors sought a "plug-and-play" solution that works with any off-the-shelf LLM during inference.

Methods

The authors introduce BLooP, a decoding-time intervention that operates entirely in the vocabulary space.

  1. Bigram Cache: Before generation begins, the system creates a hash table of every bigram (two-word sequence) present in the source document.
  2. Lookahead Promotion: During each step of the decoding process, the model looks at the previously generated token ($s_{t-1}$) and all potential next tokens ($v$).
  3. The "Nudge": If the pair $(s_{t-1}, v)$ exists in the source document's bigram cache, BLooP adds a small positive constant ($\alpha$) to the logit (raw score) of token $v$.
  4. Natural Termination: To ensure the model doesn't get "stuck" in a loop of copying, the promotion is disabled if the model predicts a newline or end-of-sequence token.

This approach is "training-free" because it doesn't change the model's weights; it simply biases the model's existing generation capabilities toward factual sequences already present in the input.

Datasets

The researchers evaluated BLooP across several diverse summarization benchmarks:

  • CNN/DM & Multi-News: Standard news summarization datasets.
  • CCSum: A newer, high-quality news dataset designed to reduce errors found in older benchmarks.
  • SciTLDR: A scientific paper summarization task, tested using abstracts, "AIC" (Abstract, Intro, Conclusion), and full-text inputs.

Key Insights and Ideas

  • Bigrams are the "Goldilocks" Zone: The authors found that boosting single words (unigrams) is too noisy, while boosting three-word sequences (trigrams) is too restrictive. Bigrams provide the perfect balance—they are specific enough to ground the model in facts but flexible enough to allow for abstractive writing.
  • Precision over Volume: BLooP only changes the model's top choice (the "argmax") in about 7% to 15% of generation steps. However, these few precise changes are enough to steer the entire summary toward higher faithfulness.
  • Targeting the "Meat": Analysis showed that BLooP primarily influences nouns and verbs. It leaves "function words" (like the, and, of) alone, meaning it preserves the model's grammatical structure while enforcing factual content.
  • Frequency is Irrelevant: Interestingly, weighting the boost based on how often a bigram appeared in the source didn't help. Simply knowing a bigram exists in the source is the most useful signal for grounding.

Key Results

The researchers tested BLooP on Llama-3.1-8B, Mistral-Nemo-12B, and Gemma-2-9B.

  • Automated Metrics: BLooP consistently improved ROUGE (lexical overlap) and BARTScore (semantic similarity) across nearly all datasets. On the CCSum dataset, Llama-3.1-8B with BLooP nearly matched the performance of the much larger GPT-3.5 (175B).
  • Human Evaluation: In a blind test, human annotators found that BLooP-augmented summaries were significantly more faithful to the source text. Crucially, there was zero loss in readability, proving that the "nudging" doesn't make the text feel robotic or disjointed.
  • Efficiency: Because it uses a simple hash table lookup, BLooP adds negligible overhead to inference time.

Limitations and Future Work

  • Long Documents: For extremely long documents (like full scientific papers), the bigram cache can become very large. The authors suggest that future versions might need to "prune" the cache to keep only the most important bigrams.
  • The $\alpha$ Parameter: Currently, the boost value ($\alpha$) is a fixed constant. The authors note that a dynamic $\alpha$ that adjusts based on the model's confidence or the specific context could potentially yield even better results.
  • Abstractiveness Trade-off: While BLooP makes summaries more factual, it does make them slightly less "novel" (i.e., the model uses fewer original phrases). The authors argue this is a favorable trade-off for applications where accuracy is paramount.

Summary updated

Abstract

Abstractive summarization requires models to generate summaries that convey information in the source document. While large language models can generate summaries without fine-tuning, they often miss key details and include extraneous information. We propose BLooP (Bigram Lookahead Promotion), a simple training-free decoding intervention that encourages large language models (LLMs) to generate tokens that form bigrams from the source document. BLooP operates through a hash table lookup at each decoding step, requiring no training, fine-tuning, or model modification. We demonstrate improvements in ROUGE and BARTScore for Llama-3.1-8B-Instruct, Mistral-Nemo-Instruct-2407, and Gemma-2-9b-it on CNN/DM, CCSum, Multi-News, and SciTLDR. Human evaluation shows that BLooP significantly improves faithfulness without reducing readability. We make the code available at https://github.com/varuniyer/BLooP
BibTeX
@inproceedings{iyer2026bloop,
  author = {Varun Iyer and Cornelia Caragea},
  title = {{BLooP: Zero-Shot Abstractive Summarization using Large Language Models with Bigram Lookahead Promotion}},
  booktitle = {LREC},
  year = {2026},
  doi = {10.63317/53bjejyjyh8r},
}
Figure