Self-Ensemble of N-best Generation Hypotheses by Lexically Constrained Decoding

Ryota Miyano, Tomoyuki Kajiwara, Yuki AraseEMNLP 2023 · DOI

Introduction

Standard text generation often fails to produce a single perfect sentence, even if the "best" parts are scattered across multiple candidates. This paper introduces a "self-ensemble" approach that identifies high-quality fragments from an $N$-best list and merges them into a superior final output using a token-level quality estimator and lexically constrained decoding.

In natural language generation (NLG), beam search is the standard algorithm for finding the most likely sequence of words. However, it suffers from the "beam search curse": increasing the beam size (searching more candidates) often leads to worse quality because the model's probability scores don't perfectly align with human-perceived quality.

Existing remedies usually involve reranking, where a second model picks the single best sentence from an $N$-best list. The authors argue this is limited: it assumes a single "perfect" hypothesis exists in the list. In reality, many candidates might be "partly right"—containing excellent phrases but failing as a complete sentence. The challenge is how to "harvest" these high-quality fragments and stitch them together into a coherent whole.

Methodology: The Self-Ensemble Pipeline

The proposed method follows a four-step pipeline to extract and recompose high-quality fragments:

  1. Generate $N$-best Hypotheses: Use a standard pre-trained model (like BART or GPT-2) to generate a list of $N$ candidate sentences.
  2. Token-level Quality Estimation (QE): This is the core innovation. The authors fine-tune a RoBERTa model to act as a binary classifier. For every token in every candidate sentence, the QE model predicts whether that token should or should not be in the final output.
  3. Constraint Composition: The system aggregates these token-level predictions across the $N$-best list using majority voting. Tokens predicted as "good" become positive lexical constraints (must include), while those predicted as "bad" become negative lexical constraints (must avoid).
  4. Lexically Constrained Decoding: Finally, the original generation model performs a second pass. Instead of standard beam search, it uses NeuroLogic-A*, an algorithm designed to satisfy lexical constraints while maintaining high sequence probability.
The proposed self-ensemble pipeline using token-level quality estimation and lexically constrained decoding.

Experimental Setup

The authors evaluated their method across three distinct NLG tasks:

  • Paraphrasing: The DIRECT corpus, which involves transforming indirect responses into direct ones (and vice versa) in conversational contexts.
  • Summarization: The CNN/Daily Mail (longer summaries) and XSum (extreme, one-sentence summaries) datasets.
  • Constrained Text Generation: CommonGen, where the model must generate a coherent sentence using a specific set of concept words.

Key Results and Insights

The proposed method consistently outperformed strong reranking baselines (like DrNMT and NCD) across all tasks.

Performance on Paraphrasing

On the DIRECT dataset (Indirect-to-Direct), the method achieved a 36.95 BLEU score, significantly higher than the beam search baseline (35.57) and the best reranker (36.01).

Performance comparison on the DIRECT dataset, highlighting the significant gap between current performance and oracle potential.

The Oracle Gap

The most striking insight comes from the "Oracle" experiments (where the system is told exactly which tokens are correct). As shown in the results table, the Oracle BLEU score jumps from ~36 to ~65. This indicates that the bottleneck isn't the decoding algorithm or the base model's knowledge, but rather the accuracy of the Quality Estimation.

Summarization and Constrained Generation

  • Summarization: On CNN/Daily Mail, it reached 41.99 ROUGE-L, surpassing the beam search baseline of 40.99.
  • Constrained Generation: On CommonGen, the method achieved a 15.62 CIDEr score, outperforming SELF-CORRECT, a state-of-the-art iterative editing method (15.30).

Discussion and Future Work

  • The "Partly High-Quality" Hypothesis: The paper successfully demonstrates that $N$-best lists contain better information at the fragment level than at the sentence level. By breaking the "one-best-sentence" constraint, they unlock higher potential quality.
  • QE as a Filter: Using a masked language model (RoBERTa) for QE allows the system to understand synonyms and multi-word expressions, helping it recognize "good" fragments even if they don't exactly match a single reference.
  • Limitations: The method is slower than standard decoding because it requires two passes. Additionally, the QE model is far from perfect, recalling only 31-41% of correct tokens. Improving token-level QE and implementing confidence-based constraint weighting are promising paths for future gains.

Summary updated

BibTeX
@inproceedings{miyano2023self,
  author = {Ryota Miyano and Tomoyuki Kajiwara and Yuki Arase},
  title = {{Self-Ensemble of N-best Generation Hypotheses by Lexically Constrained Decoding}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.emnlp-main.905},
}
Figure