Controllable Text Generation for Open-Domain Creativity and Fairness

Nanyun (Violet) Peng

Introduction

While large pre-trained language models (LLMs) have achieved remarkable success in constrained tasks such as machine translation and summarization, they often struggle in open-domain scenarios like storytelling or poetry. These tasks are characterized by high conditional entropy, where a single prompt can lead to an infinite variety of valid continuations. In these settings, standard auto-regressive models—which predict the next token based on surface-level patterns—frequently suffer from a lack of long-term coherence and the amplification of social biases present in their training data. This paper outlines a research program dedicated to moving beyond simple next-token prediction toward a framework of controllable text generation that prioritizes discourse structure, literary aesthetics, and ethical alignment.

Methodology: From Surface Patterns to Structural Control

The core of the proposed approach is the disentanglement of high-level planning from low-level surface realization. By treating creativity as a structural challenge rather than a purely linguistic one, the research introduces several key strategies:

1. Hierarchical Plan-and-Write Framework

Instead of generating text word-by-word from scratch, the system first generates a "structured plan" consisting of entities, events, and relations. A separate surface realization model then converts this outline into natural language. This hierarchical approach allows for human-in-the-loop collaboration at the planning stage and ensures the model maintains a consistent "roadmap" throughout long-form generation.

2. Temporal Modeling and Non-linear Narratives

To move beyond repetitive or boring linear plots, the research utilizes temporal prompts (e.g., tags like <before> and <after>) and rescoring models. These tools allow the system to re-rank naive plans based on their temporal logic and relevance, enabling the generation of sophisticated narrative devices such as flashbacks.

3. Insertion-based Generation

Traditional models are restricted to left-to-right generation. The authors propose an insertion-based framework that allows models to "fill in the blanks" or expand upon a plan non-monotonically. This flexibility is particularly effective for satisfying specific lexical constraints and refining text iteratively.

4. Controllable Decoding for Fairness

Rather than relying on the expensive fine-tuning of massive models, this research emphasizes post-processing techniques. By using small auxiliary models or specialized decoding algorithms, the system can guide the LLM toward fairer outputs during inference. This approach treats bias mitigation as a controllable generation problem, effectively "guardrailing" the model against toxic language and social stereotypes without sacrificing its creative potential.

Evaluation and Datasets

The research draws on a diverse array of data sources and benchmarks to validate these methods:

  • Silver Standard Plans: Extracted from large corpora using information extraction (IE) tools to identify underlying event structures.
  • Creative Benchmarks: Evaluation across domains including sonnets, poetry, and figurative language (puns, similes, and metaphors).
  • Fairness Metrics: The development of automated metrics to detect social stereotypes and ad hominem attacks in open-domain dialogue systems.

Key Insights and Results

The research demonstrates that explicitly modeling the structure of a story solves the "drifting" problem common in long-form LLM outputs. Key findings include:

  • Enhanced Coherence: Hierarchical models significantly outperform flat, auto-regressive models in maintaining plot consistency over long distances.
  • Constraint Fidelity: Specialized attention mechanisms and insertion-based models show much higher fidelity to user-provided keywords, whereas traditional models often fail to include up to 20% of required plan elements.
  • Zero-Shot Creativity: The framework successfully generates sonnets with discourse-level coherence and poetic features in zero-shot settings.
  • Effective Bias Reduction: Controllable decoding methods successfully reduced the presence of harmful stereotypes in "chit-chat" dialogue without degrading the fluency of the conversation.

Limitations and Future Directions

Despite these advancements, several challenges remain. Insertion-based models currently require scaling to larger datasets to match the raw performance of standard auto-regressive LLMs. Furthermore, there is a persistent need for better integration of external common sense knowledge bases to ensure that creative outputs remain logically "sensible." Future work aims to create a virtuous cycle by using generation models to improve information extraction tools, which in turn will provide better structured data for future planning models.

Summary updated

BibTeX
@inproceedings{peng2022controllable,
  author = {Nanyun (Violet) Peng},
  title = {{Controllable Text Generation for Open-Domain Creativity and Fairness}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/818},
}
Figure