Adapting a Language Model for Controlled Affective Text Generation

Tushar Goswamy, Ishika Singh, Ahsan Barkati, Ashutosh Modi

Introduction

Modern large language models (LLMs) excel at generating fluent, grammatically correct text by predicting the next most likely token. However, this optimization often results in output that is emotionally flat or "monotonous." Previous attempts at affective text generation—steering models toward specific emotions—have historically faced a significant "Grammar-Intensity Trade-off." As the emotional intensity increases, the linguistic coherence often degrades into "word salad." Furthermore, many existing systems offer only coarse control, limited to binary sentiment (positive/negative) rather than a nuanced spectrum of human emotions.

Researchers at IIT Kanpur address these challenges by introducing a controllable "knob" for emotional intensity. Their approach allows for the generation of text that is both emotionally charged and grammatically sound, supporting a wide range of emotions with fine-grained intensity control.

Methods: The Affect Knob and Latent Perturbation

The proposed model builds upon the Plug and Play Language Model (PPLM) framework, utilizing GPT-2 Medium as the base architecture. The core innovation lies in a novel Gaussian-based loss function designed to "steer" the generation process toward a target emotional intensity.

Overview of the affective text generation pipeline, featuring the intensity "knob" (beta) and Plutchik’s Wheel of Emotion.

Key components of the methodology include:

  • The Affect Knob ($\beta$): Instead of a binary toggle, the model uses a Gaussian function $N(affectInt, knob, var)$, where the "knob" represents the mean intensity. Users can specify a target intensity level on a scale from 0 to 1.
  • Latent Perturbation: Rather than simple post-generation filtering, the model perturbs the "history matrix" (internal memory) during the generation of each word. This ensures that emotional intent is structurally integrated into the sentence.
  • Emotional Framework: The model supports eight basic emotions—Joy, Trust, Fear, Surprise, Sadness, Disgust, Anger, and Anticipation—based on Plutchik’s psychological theory.
  • Lexical Guidance: The NRC Emotion Intensity Lexicon, containing approximately 10,000 human-annotated words, provides the necessary intensity scores to guide the loss function.

Key Insights: Resilience and Independence

A primary finding of this work is that grammar is resilient to emotional steering. By incorporating a KL-Divergence term in the loss function, the model is constrained to remain near the original "natural" language distribution of GPT-2 while moving toward the desired affective target. This prevents the fluency collapse seen in earlier models.

Additionally, the architecture demonstrates Topic-Affect Independence. The model successfully decouples the subject matter (e.g., "Technology") from the emotional delivery (e.g., "Joy" vs. "Fear"), allowing for versatile application across different domains.

Results and Evaluation

The model was evaluated against AffectLM (an LSTM-based model) and the original PPLM.

Comparison of Perplexity (automated) and Grammatical Correctness (human evaluation) across different models and intensity settings.
  • Grammaticality and Fluency: As shown in Figure 5, the proposed model maintains stable perplexity and high human-rated grammatical correctness even at maximum intensity. In contrast, PPLM’s fluency significantly degrades at high settings.
  • Emotion Accuracy: In human trials for a 3-emotion classification task at moderate intensity (knob=0.4), the model achieved 83.3% accuracy, vastly outperforming AffectLM’s 39.2%.
  • Intensity Control: Human annotators confirmed a strong correlation between the "knob" value and the perceived intensity of emotions like Anger, Sadness, and Joy.

Conclusion and Future Work

While the model provides unprecedented control, some challenges remain. Subtle distinctions between emotions like "Fear" and "Surprise" can be difficult for both the model to generate and humans to distinguish. Furthermore, at extreme intensities (knob=1.0), human classification accuracy dips slightly, suggesting that overt emotional signaling may occasionally obscure context.

Future work involves integrating this controlled affective generation into interactive virtual agents and conversational therapy bots, where empathetic, nuanced, and reliable emotional responses are essential.

Summary updated

BibTeX
@inproceedings{goswamy2020adapting,
  author = {Tushar Goswamy and Ishika Singh and Ahsan Barkati and Ashutosh Modi},
  title = {{Adapting a Language Model for Controlled Affective Text Generation}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2020},
  url = {https://openreview.net/forum?id=H3G-WSmX8kc},
}
Figure