Unsupervised Neural Stylistic Text Generation using Transfer learning and Adapters

Vinayshekhar Bannihatti Kumar, Rashmi Gangadharaiah, Dan RotharXiv 2022 · DOI · arXiv

Abstract

Generating text with specific personas or styles typically requires expensive parallel datasets. Researchers at AWS AI Labs have developed a novel three-phase training framework called Learn, Learn, and Relearn (LLR) that injects stylistic attributes into text generation models using unsupervised methods. By updating only 0.3% of the model’s parameters via adapters, this approach improves stylistic accuracy by over 200% compared to standard baselines while maintaining high content relevance.

Introduction: The Personality Crisis in Text Generation

Most state-of-the-art text generation models suffer from a "personality crisis." Trained on massive, crowdsourced datasets, they tend to adopt an "average" style—a bland, robotic middle ground. While conversational agents benefit from distinct personas (e.g., "cheerful" or "skeptical"), training them usually requires parallel corpora (the same sentence in multiple styles), which are difficult to acquire.

Existing unsupervised methods often face a "style-relevance trade-off": increasing stylistic intensity often causes the model to lose track of the conversation context.

Example of stylistic story endings for a given context.

Methodology: The LLR Framework

The proposed LLR (Learn, Learn, and Relearn) procedure utilizes a BERT encoder and a GPT-2 decoder. The core innovation is the use of Adapters—tiny, task-specific layers inserted into the pre-trained model. Instead of retraining the entire model, only these small layers (approximately 0.3% of total parameters) are updated.

The three-phase LLR training framework: Learn (Phase 1), Learn (Phase 2), and Relearn (Phase 3).

The training consists of three distinct phases:

  1. Learn (Phase 1): The base Encoder-Decoder model is fine-tuned on a standard task (e.g., story ending generation) to ensure coherent content generation.
  2. Learn (Phase 2): The decoder is detached, and only the adapter weights are trained on a style-specific corpus (e.g., "Negative" text) using a language modeling objective.
  3. Relearn (Phase 3): The model is reconnected, and adapter weights are briefly retrained on the original task. This "aligns" the learned style with the requirement to remain contextually relevant.

Experimental Setup and Datasets

The framework was evaluated using two primary datasets:

  • ROC Stories Corpus: A collection of 5-sentence stories used for generating logical endings.
  • PERSONALITY-CAPTIONS Dataset: Containing over 241,000 image captions in 215 styles. The researchers clustered these into broader categories (e.g., grouping "Arrogant" and "Gloomy" into a "Negative" style) to create style-specific corpora.

Key Insights and Results

The "Relearn" phase proved critical; without it, stylistic adapters often produced nonsensical gibberish. Phase 3 acts as a sanity check, forcing stylistic choices to respect the input context.

Performance Metrics

  • Style Accuracy: The LLR model achieved a 200% improvement in style generation, measured by the "Ratio of endings In Style" (RIS) metric.
  • Content Relevance: Despite the stylistic shift, the model maintained relevance scores nearly identical to non-stylistic baselines, with only a 9% drop.
  • Adapter Architectures: Testing various architectures revealed that Invertible Adapters performed best, likely due to their ability to capture complex language transformations.

Human Validation

In a pilot study, human judges identified LLR-generated endings as significantly more stylistic (e.g., "sad") than baselines while rating them as equally relevant to the story context.

Limitations and Future Directions

While stylistically proficient, the model occasionally lacks common-sense grounding. For instance, a "money-minded" persona might inappropriately associate a dog with "stolen property" due to a lack of deep world knowledge. Future research will explore integrating common-sense reasoning—potentially via graph attention networks—to ensure stylistic choices remain logically sound.

Summary updated

Abstract

Research has shown that personality is a key driver to improve engagement and user experience in conversational systems. Conversational agents should also maintain a consistent persona to have an engaging conversation with a user. However, text generation datasets are often crowd sourced and thereby have an averaging effect where the style of the generation model is an average style of all the crowd workers that have contributed to the dataset. While one can collect persona-specific datasets for each task, it would be an expensive and time consuming annotation effort. In this work, we propose a novel transfer learning framework which updates only $0.3\%$ of model parameters to learn style specific attributes for response generation. For the purpose of this study, we tackle the problem of stylistic story ending generation using the ROC stories Corpus. We learn style specific attributes from the PERSONALITY-CAPTIONS dataset. Through extensive experiments and evaluation metrics we show that our novel training procedure can improve the style generation by 200 over Encoder-Decoder baselines while maintaining on-par content relevance metrics with
BibTeX
@misc{kumar2022unsupervised,
  author = {Vinayshekhar Bannihatti Kumar and Rashmi Gangadharaiah and Dan Roth},
  title = {{Unsupervised Neural Stylistic Text Generation using Transfer learning and Adapters}},
  year = {2022},
  eprint = {2210.03264},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2210.03264},
}
Figure