Evaluating the Prompt Steerability of Large Language Models

Erik Miehling, Michael Desmond, Karthikeyan Natesan Ramamurthy, Elizabeth M. Daly, Kush R. Varshney, Eitan Farchi, Pierre Dognin, Jesus Rios, Djallel Bouneffouf, Miao Liu, Prasanna SattigeriNAACL 2025 · DOI

Introduction

As Large Language Models (LLMs) become increasingly integrated into society, the focus of AI alignment is shifting. While traditional research aims to align models with a singular set of "average" human preferences, the concept of AI pluralism suggests that models should be capable of representing a diverse array of value systems, cultures, and personas. To achieve this, we must understand and quantify "steerability"—the ease with which a model's behavior can be shifted through simple prompting.

Methodology: Quantifying Behavioral Shifts

The researchers introduce a formal framework based on Evaluation Profiles to measure steerability. The process begins with Base Profiling, where a model's natural behavior is assessed on specific dimensions (e.g., agreeableness) using yes/no questions without special instructions.

Next, the model is subjected to Prompt Steering. A "steering budget" ($k$) of principles is added to the system prompt to nudge the model toward a specific persona. To quantify the resulting change, the authors utilize Steerability Indices derived from the Wasserstein distance.

Visualization of the Wasserstein distance used to calculate steerability indices.

This metric is particularly effective because it accounts for the model's baseline, measuring how much of the "available room" for steering is actually utilized. By plotting these indices against the steering budget $k$, the researchers generate Steerability Curves that visualize the dynamics of behavioral change.

Key Insights

The study, utilizing 32 high-confidence dimensions from Anthropic’s persona dataset, revealed several critical findings regarding how models respond to steering.

The Plateau Effect and Model Sophistication

Advanced models, such as Llama-3.1-8b and Phi-3-medium, demonstrate higher steerability. They often reach a performance plateau quickly, requiring only a small steering budget to adopt a new persona. In contrast, less sophisticated models may require significantly more principles to show a comparable shift.

Steerability curves for the 'ends-justify-means' dimension.

Steering Asymmetry and Base Profiles

Models are not blank slates; they possess inherent "base profiles" shaped by pre-training and RLHF. This leads to steering asymmetry, where it is often easier to steer a model toward one extreme of a trait than the other. For instance, if a model is naturally conscientious, nudging it to be "anti-conscientious" is straightforward, while making it more conscientious may be nearly impossible as it is already near the ceiling.

Non-Monotonicity

Interestingly, increasing the steering budget does not always result in more steering. Some models exhibited regressive behavior after a certain point, potentially due to the activation of internal safety filters or defenses against long-context "jailbreaking" attempts.

Results and Model Comparison

The benchmark indicates that models are most flexible regarding Ethics/Philosophy and Personality, while showing significant resistance in Politics and Risk-taking. Across most dimensions, a "negative bias" was observed: models were generally easier to steer toward negative traits (e.g., cynicism) than positive ones. In direct comparisons, Llama-3.1-8b and Phi-3-medium-4k-instruct proved to be the most steerable, outperforming earlier Llama versions and IBM's Granite models.

Conclusion

While this benchmark provides a robust quantitative tool, it has limitations. The high number of model calls required makes it computationally expensive, and the current focus is limited to single-turn interactions. Future work will explore how personas evolve in multi-turn conversations and investigate the link between In-Context Learning (ICL) and steerability. Understanding this connection may be the key to developing truly pluralistic AI systems.

Summary updated

BibTeX
@inproceedings{miehling2025evaluating,
  author = {Erik Miehling and Michael Desmond and Karthikeyan Natesan Ramamurthy and Elizabeth M. Daly and Kush R. Varshney and Eitan Farchi and Pierre Dognin and Jesus Rios and Djallel Bouneffouf and Miao Liu and Prasanna Sattigeri},
  title = {{Evaluating the Prompt Steerability of Large Language Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-long.400},
}
Figure