Gradient-guided Unsupervised Lexically Constrained Text Generation

Lei ShaEMNLP 2020 · OpenReview

Abstract

Lexically constrained generation requires target sentences to satisfy specific constraints, such as including mandatory keywords or preserving semantic meaning during paraphrasing. Traditional methods relying on beam search or stochastic sampling often struggle with efficiency and fluency in large search spaces. We introduce G2LC (Gradient-guided Lexically Constrained generation), which reformulates constrained generation as an unsupervised optimization problem. By utilizing a differentiable objective function, G2LC leverages gradients to pinpoint optimal edit positions and refine word selection. Our results on keyword-to-sentence and unsupervised paraphrase generation tasks demonstrate that G2LC achieves state-of-the-art performance, offering a flexible, training-free solution for constrained text generation.

Introduction

Controlling the output of natural language generation models is essential for practical applications, ranging from automated summarization with mandatory keywords to high-quality paraphrasing. These "lexical constraints" are typically categorized as hard constraints (specific tokens) or soft constraints (semantic similarity).

Existing techniques face significant hurdles. Beam search is effective for narrow tasks but fails to scale when constraints require searching through vast sentence spaces. Conversely, stochastic search methods like MCMC offer flexibility but are hindered by random "trial-and-error" edits, which often result in nonsensical or disfluent outputs. G2LC overcomes these limitations by introducing gradient-based guidance, providing a "compass" for the editing process without requiring parallel training data.

Methodology

The G2LC framework treats constrained generation as an optimization problem in a continuous space. The core innovation lies in making the discrete text generation process differentiable.

1. Differentiable Objective Functions

To enable gradient descent, the model's objectives must be smooth. G2LC employs a sliding window approach to measure lexical constraint satisfaction and adapts standard language models to calculate fluency probabilities based on embedding distances. For paraphrasing, a separate recognition model ensures semantic consistency.

2. Gradient-Guided Editing

The model identifies the most effective position to edit by calculating the gradient of the loss function with respect to each word representation. The position with the largest gradient norm is flagged as the "most urgent" site for modification (insertion, deletion, or replacement).

Overview of the G2LC framework: using gradient norms to identify the optimal edit position in a sequence.

3. Fuzzy Word Refinement

When a new word is required, G2LC starts with a "fuzzy" embedding—a weighted average of potential candidates. It then uses gradient descent to fine-tune this embedding before "snapping" it to the nearest actual word in the vocabulary, ensuring both constraint satisfaction and local fluency.

Experimental Evaluation

The authors evaluated G2LC across two primary benchmarks: Keyword-to-Sentence Generation (using the One-Billion Word Corpus) and Unsupervised Paraphrase Generation (using the Quora Question Pair and SNLI datasets).

Keyword-to-Sentence Generation

G2LC demonstrated superior fluency, achieving lower Negative Log-Likelihood (NLL) scores than previous state-of-the-art methods like CGMH and Grid Beam Search. Human evaluations corroborated these metrics, with G2LC consistently rated higher for relevance and naturalness across varying numbers of keywords (1–4).

Unsupervised Paraphrase Generation

In the paraphrase task, G2LC significantly outperformed other unsupervised baselines. On the Quora dataset, G2LC (Generator) achieved a BLEU score of 23.27, nearly doubling the performance of VAE (13.96) and outperforming CGMH (15.73). Notably, G2LC's performance rivaled some supervised models, highlighting the power of gradient-guided search in unsupervised settings.

Performance comparison on unsupervised paraphrase generation, showing G2LC outperforming other unsupervised baselines.

Conclusion

G2LC represents a significant advancement in lexically constrained text generation. By bridging the gap between continuous optimization and discrete word selection, it provides a principled way to navigate complex search spaces. Its unsupervised nature and "plug-and-play" compatibility with existing pre-trained models make it a versatile tool for real-world NLG applications. Future research will focus on optimizing inference speed and exploring more complex structural constraints.

Summary updated

Abstract

Lexically constrained generation requires the target sentence to satisfy some lexical constraints, such as containing some specific words or being the paraphrase to a given sentence, which is very important in many real-world natural language generation applications. Previous works usually apply beam-search-based methods or stochastic searching methods to lexically-constrained generation. However, when the search space is too large, beam-search-based methods always fail to find the constrained optimal solution. At the same time, stochastic search methods always cost too many steps to find the correct optimization direction. In this paper, we propose a novel method G2LC to solve the lexically-constrained generation as an unsupervised gradient-guided optimization problem. We propose a differentiable objective function and use the gradient to help determine which position in the sequence should be changed (deleted or inserted/replaced by another word). The word updating process of the inserted/replaced word also benefits from the guidance of gradient. Besides, our method is free of parallel data training, which is flexible to be used in the inference stage of any pre-trained generation model. We apply G2LC to two generation tasks: keyword-to-sentence generation and unsupervised paraphrase generation. The experiment results show that our method achieves state-of-the-art compared to previous lexically-constrained methods.
BibTeX
@inproceedings{sha2020gradient,
  author = {Lei Sha},
  title = {{Gradient-guided Unsupervised Lexically Constrained Text Generation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=WZBGGBgaw9},
}
Figure