Abstract
Generating text with specific personas or styles typically requires expensive parallel datasets. Researchers at AWS AI Labs have developed a novel three-phase training framework called Learn, Learn, and Relearn (LLR) that injects stylistic attributes into text generation models using unsupervised methods. By updating only 0.3% of the model’s parameters via adapters, this approach improves stylistic accuracy by over 200% compared to standard baselines while maintaining high content relevance.
Introduction: The Personality Crisis in Text Generation
Most state-of-the-art text generation models suffer from a "personality crisis." Trained on massive, crowdsourced datasets, they tend to adopt an "average" style—a bland, robotic middle ground. While conversational agents benefit from distinct personas (e.g., "cheerful" or "skeptical"), training them usually requires parallel corpora (the same sentence in multiple styles), which are difficult to acquire.
Existing unsupervised methods often face a "style-relevance trade-off": increasing stylistic intensity often causes the model to lose track of the conversation context.
Methodology: The LLR Framework
The proposed LLR (Learn, Learn, and Relearn) procedure utilizes a BERT encoder and a GPT-2 decoder. The core innovation is the use of Adapters—tiny, task-specific layers inserted into the pre-trained model. Instead of retraining the entire model, only these small layers (approximately 0.3% of total parameters) are updated.
The training consists of three distinct phases:
- Learn (Phase 1): The base Encoder-Decoder model is fine-tuned on a standard task (e.g., story ending generation) to ensure coherent content generation.
- Learn (Phase 2): The decoder is detached, and only the adapter weights are trained on a style-specific corpus (e.g., "Negative" text) using a language modeling objective.
- Relearn (Phase 3): The model is reconnected, and adapter weights are briefly retrained on the original task. This "aligns" the learned style with the requirement to remain contextually relevant.
Experimental Setup and Datasets
The framework was evaluated using two primary datasets:
- ROC Stories Corpus: A collection of 5-sentence stories used for generating logical endings.
- PERSONALITY-CAPTIONS Dataset: Containing over 241,000 image captions in 215 styles. The researchers clustered these into broader categories (e.g., grouping "Arrogant" and "Gloomy" into a "Negative" style) to create style-specific corpora.
Key Insights and Results
The "Relearn" phase proved critical; without it, stylistic adapters often produced nonsensical gibberish. Phase 3 acts as a sanity check, forcing stylistic choices to respect the input context.
Performance Metrics
- Style Accuracy: The LLR model achieved a 200% improvement in style generation, measured by the "Ratio of endings In Style" (RIS) metric.
- Content Relevance: Despite the stylistic shift, the model maintained relevance scores nearly identical to non-stylistic baselines, with only a 9% drop.
- Adapter Architectures: Testing various architectures revealed that Invertible Adapters performed best, likely due to their ability to capture complex language transformations.
Human Validation
In a pilot study, human judges identified LLR-generated endings as significantly more stylistic (e.g., "sad") than baselines while rating them as equally relevant to the story context.
Limitations and Future Directions
While stylistically proficient, the model occasionally lacks common-sense grounding. For instance, a "money-minded" persona might inappropriately associate a dog with "stolen property" due to a lack of deep world knowledge. Future research will explore integrating common-sense reasoning—potentially via graph attention networks—to ensure stylistic choices remain logically sound.
Summary updated