Abstract
Pretrained Transformer-based language models (LMs) display remarkable natural language generation capabilities. With their immense potential, controlling text generation of such LMs is getting attention. While there are studies that seek to control high-level attributes (such as sentiment and topic) of generated text, there is still a lack of more precise control over its content at the word- and phrase-level. Here, we propose Content-Conditioner (CoCon) to control an LM's output text with a content input, at a fine-grained level. In our self-supervised approach, the CoCon block learns to help the LM complete a partially-observed text sequence by conditioning with content inputs that are withheld from the LM. Through experiments, we show that CoCon can naturally incorporate target content into generated texts and control high-level text attributes in a zero-shot manner.
Introduction
Large-scale Transformer models, such as GPT-2, have revolutionized natural language generation with their fluency. However, these models are notoriously difficult to steer toward specific outputs. While existing Controlled Text Generation (CTG) methods effectively manage global attributes like sentiment or broad topics, they lack the precision required for word- or phrase-level control. Users often need a model to incorporate specific information into a fluent output without the prohibitive cost of full model retraining. CoCon addresses this gap by introducing a modular, self-supervised layer designed for fine-grained content steering.
Methodology
Architecture
The Content-Conditioner (CoCon) is a lightweight Transformer block interleaved between the layers of a frozen, pretrained language model. The architecture divides the original LM into two functional components:
- $LM_\alpha$ (The Encoder): Extracts initial representations from the input prompt.
- CoCon Block: Blends the prompt representations with a specific content input using an attention mechanism and a feed-forward network.
- $LM_\beta$ (The Decoder): Generates the final text based on the conditioned representations.
Self-Supervised Training
CoCon is trained without human-labeled data using a self-reconstruction objective. A natural text sequence is split into two segments ($x^a$ and $x^b$). The model uses $x^a$ as the prompt and $x^b$ as the target content, learning to reconstruct the original full sequence. To improve robustness when prompt and content come from different sources, the authors employ a Cycle Reconstruction Loss, ensuring content integrity is maintained across generation steps.
Experimental Setup
The model was primarily trained on 250,000 GPT-2 generated samples, with additional testing on the WebText dataset. Evaluation utilized the IMDB Movie Reviews for sentiment classification and the HuffPost News Category Dataset for topic classification (covering Politics, Science, Religion, and Computers).
Results and Discussion
CoCon demonstrates superior performance over baselines like PPLM and CTRL in both relevance and attribute accuracy.
- Topic and Sentiment Control: CoCon achieved 90.4% accuracy in topic steering (compared to 42.5% for PPLM and 86.7% for CTRL) and 98.9% accuracy for sentiment steering.
- Zero-Shot Capability: Despite being trained on content similarity, CoCon can steer high-level attributes in a zero-shot manner by using attribute-specific words (e.g., "Scientist") as content inputs.
- Controllable Strength: An inference-time strength parameter ($\tau$) allows users to adjust the influence of the content input. However, the authors noted a fluency trade-off: excessively high values can degrade output into repetitive "word salad."
Conclusion
CoCon provides a modular and efficient "steering wheel" for pretrained language models. By leveraging self-supervised reconstruction, it achieves fine-grained control over generated content while maintaining competitive fluency. Future work may explore extending CoCon to incorporate structured knowledge graphs or more complex stylistic constraints.
Summary updated