Abstract
This paper proposes a simple method for controllable text generation based on weighting logits with a free-form classifier, namely CAIF sampling. Using an arbitrary text classifier, we adjust a small part of a language model's logits and guide text generation towards or away from classifier prediction. We experimented with toxicity avoidance and sentiment control tasks and showed that the proposed method significantly outperforms recent PPLM, GeDi, and DExperts on PPL and task accuracy metrics based on the external classifier of generated texts. In addition, compared to other approaches, it is easier to implement and tune and has significantly fewer restrictions and requirements.
Introduction
Large Language Models (LLMs) are powerful but notoriously difficult to "steer." Current methods for guiding generation toward specific attributes like non-toxicity or positive sentiment often face a "scarcity gap." While thousands of text classifiers are available on platforms like Hugging Face, specialized conditional language models—required by methods like GeDi or DExperts—are rare. Furthermore, existing classifier-based methods like PPLM are computationally intensive due to backpropagation requirements through the LM's hidden states.
The CAIF Sampling Method
The authors introduce CAIF (Classifier guided sampling), which leverages standard text classifiers for steering. The core algorithm follows a four-step process for each token generation:
- Logit Filtering: The LM identifies the top-$j$ most likely tokens (typically $j \approx 100$).
- Classifier Guidance: An external, arbitrary classifier evaluates these $j$ tokens.
- Bayesian Re-weighting: Original probabilities are adjusted by the classifier's prediction raised to a hyperparameter $\alpha$.
- Sampling: The final token is sampled from the adjusted distribution.
Efficiency via Entropy Triggering
To mitigate the overhead of running a classifier at every step, the authors propose an Entropy Criterion. They observed that guidance is most critical when the LM is "uncertain" (high entropy), whereas low-entropy steps often correspond to utilitarian or grammatical tokens (e.g., "the", "and"). By only triggering the classifier when the LM's output entropy exceeds a predefined threshold, CAIF maintains control while skipping the classifier for 50-80% of tokens, significantly reducing computational cost.
Experimental Results
CAIF was evaluated on toxicity avoidance using the RealToxicityPrompts dataset and sentiment control using the OpenWebText corpus.
- Toxicity Avoidance: CAIF reduced the probability of generating toxic text to 3.3%, outperforming DExperts (6.4%) and GeDi (11.2%).
- Fluency: CAIF achieved a perplexity (PPL) of 15.0, significantly better than DExperts (20.2) and GeDi (60.0), indicating more natural and fluent text.
- Flexibility: The use of a negative $\alpha$ proved highly effective for avoidance tasks, allowing the model to steer away from undesired classes like toxicity.
Conclusion and Limitations
CAIF provides a simple, effective, and flexible framework for controllable generation. By tapping into the vast library of existing classifiers and using entropy-based triggering, it achieves state-of-the-art performance with lower implementation complexity than previous methods. However, because the classifier must process the entire sequence prefix at each guided step, the time complexity is $O(n^2)$ relative to sequence length, making it less efficient than "expert LM" methods for very long generations. Future work may explore more sophisticated triggering criteria to further optimize speed.
Summary updated