Abstract
Text generation with sentiment control is difficult without fine-tuning or modifying the model architecture. Plug and Play Language Model (PPLM) utilizes an external sentiment classifier to update the hidden states of GPT-2 at each time step. It does not change the parameters but achieves competitive performance. However, fluency is impaired due to the instability of the hidden states. Moreover, the classifier is not strong because of the way it is trained with partial texts, hence it is difficult to guide the generation in the process. To solve the above problems, in this paper, we first propose a fixed threshold method based on the Valence-Arousal-Dominance (VAD) lexicon to decide whether to change a word, which keeps the fluency of the original LM to the greatest extent. Furthermore, for the improvement of sentiment alignment, we propose a dynamic threshold method that utilizes VAD-based loss to make the threshold dynamic. Experiments demonstrate that our methods outperform the baseline with a great margin significantly both on fluency and sentiment accuracy.
Introduction
Large-scale pre-trained language models like GPT-2 produce remarkably fluent text but are notoriously difficult to steer toward specific attributes, such as sentiment, without extensive fine-tuning. The Plug and Play Language Model (PPLM) framework offers a solution by using an external classifier to "nudge" the model's hidden states at each generation step. However, this constant intervention often leads to a "tug-of-war" between sentiment control and linguistic naturalness. The aggressive updates can destabilize hidden states, resulting in "semantic confusion"—sentences that may meet sentiment targets but lack grammatical coherence.
Methodology: The "Change or Not" Strategy
To address the instability of standard PPLM, this research introduces a "gatekeeping" mechanism that selectively applies perturbations. Instead of updating every hidden state, the system decides whether a suggested word change is worth the potential cost to fluency.
PPLM-FT (Fixed Threshold)
In this approach, the model samples two candidate words at each step: one from the original (unperturbed) distribution and one from the sentiment-steered (perturbed) distribution. The system then consults the Valence-Arousal-Dominance (VAD) Lexicon, which contains human-rated scores for approximately 20,000 English words. A change is only enacted if the difference in valence ($\Delta V$) between the two candidates exceeds a fixed threshold. If the sentiment gain is marginal, the model retains the original word, thereby preserving the base model's fluency.
PPLM-DT (Dynamic Threshold)
Recognizing that a static threshold may be too rigid, the authors developed a dynamic version. PPLM-DT incorporates a VAD-based loss function that monitors the cumulative sentiment of the generated sequence. As the sentence increasingly reflects the target sentiment, the threshold for future word changes rises. This "sentiment budgeting" allows the model to relax its control once the goal is met, letting the base language model ensure a natural conclusion to the sentence.
Experimental Setup
The methods were evaluated using the VAD Lexicon and a set of 50 distinct prompts. For each prompt, 10 samples were generated (500 total) to assess sentiment accuracy, fluency, and diversity.
Key Results
The proposed methods significantly outperformed the standard PPLM baseline across both automatic and human metrics.
- Fluency: Automatic perplexity (PPL) scores showed significant improvement, with PPLM-FT dropping to 44.14 from the baseline's 48.42. Human trials corroborated this, with fluency scores jumping from 2.95 to 3.70 on a 5-point scale.
- Sentiment Accuracy: PPLM-DT achieved the highest sentiment accuracy (ACC) at 63.73%, compared to 59.60% for the baseline.
- Diversity: Both PPLM-FT and PPLM-DT maintained high "Dist-n" scores, indicating that the selective intervention does not lead to repetitive or generic text generation.
Conclusion and Future Work
By treating sentiment control as a selective process rather than a constant pressure, the "Change or Not" approach effectively balances steerability with fluency. The use of a human-validated lexicon provides a stable ground truth that mitigates the noise of step-by-step classifiers.
Future research may address the limitations of lexicon coverage, as words outside the VAD list are currently treated as neutral. Additionally, extending this dynamic thresholding logic to more complex emotional states beyond simple valence remains a promising avenue for exploration.
Summary updated