Introduction
As Large Language Models (LLMs) become increasingly integrated into society, the focus of AI alignment is shifting. While traditional research aims to align models with a singular set of "average" human preferences, the concept of AI pluralism suggests that models should be capable of representing a diverse array of value systems, cultures, and personas. To achieve this, we must understand and quantify "steerability"—the ease with which a model's behavior can be shifted through simple prompting.
Methodology: Quantifying Behavioral Shifts
The researchers introduce a formal framework based on Evaluation Profiles to measure steerability. The process begins with Base Profiling, where a model's natural behavior is assessed on specific dimensions (e.g., agreeableness) using yes/no questions without special instructions.
Next, the model is subjected to Prompt Steering. A "steering budget" ($k$) of principles is added to the system prompt to nudge the model toward a specific persona. To quantify the resulting change, the authors utilize Steerability Indices derived from the Wasserstein distance.
This metric is particularly effective because it accounts for the model's baseline, measuring how much of the "available room" for steering is actually utilized. By plotting these indices against the steering budget $k$, the researchers generate Steerability Curves that visualize the dynamics of behavioral change.
Key Insights
The study, utilizing 32 high-confidence dimensions from Anthropic’s persona dataset, revealed several critical findings regarding how models respond to steering.
The Plateau Effect and Model Sophistication
Advanced models, such as Llama-3.1-8b and Phi-3-medium, demonstrate higher steerability. They often reach a performance plateau quickly, requiring only a small steering budget to adopt a new persona. In contrast, less sophisticated models may require significantly more principles to show a comparable shift.
Steering Asymmetry and Base Profiles
Models are not blank slates; they possess inherent "base profiles" shaped by pre-training and RLHF. This leads to steering asymmetry, where it is often easier to steer a model toward one extreme of a trait than the other. For instance, if a model is naturally conscientious, nudging it to be "anti-conscientious" is straightforward, while making it more conscientious may be nearly impossible as it is already near the ceiling.
Non-Monotonicity
Interestingly, increasing the steering budget does not always result in more steering. Some models exhibited regressive behavior after a certain point, potentially due to the activation of internal safety filters or defenses against long-context "jailbreaking" attempts.
Results and Model Comparison
The benchmark indicates that models are most flexible regarding Ethics/Philosophy and Personality, while showing significant resistance in Politics and Risk-taking. Across most dimensions, a "negative bias" was observed: models were generally easier to steer toward negative traits (e.g., cynicism) than positive ones. In direct comparisons, Llama-3.1-8b and Phi-3-medium-4k-instruct proved to be the most steerable, outperforming earlier Llama versions and IBM's Granite models.
Conclusion
While this benchmark provides a robust quantitative tool, it has limitations. The high number of model calls required makes it computationally expensive, and the current focus is limited to single-turn interactions. Future work will explore how personas evolve in multi-turn conversations and investigate the link between In-Context Learning (ICL) and steerability. Understanding this connection may be the key to developing truly pluralistic AI systems.
Summary updated