Persona Switch: Mixing Distinct Perspectives in Decoding Time

Junseok Kim, Nakyeong Yang, Kyomin Jung

TL;DR

This paper introduces "Persona Switch," a novel decoding strategy that dynamically combines zero-shot and role-play prompting for Large Language Models (LLMs). By selecting the better output at each decoding step based on confidence scores, Persona Switch significantly improves reasoning accuracy, outperforming traditional methods.

What Problem Does This Paper Address?

Large Language Models (LLMs) can be steered to improve their zero-shot reasoning capabilities by injecting a "persona" into the prompt (role-play prompting). However, the benefits of role-play prompting are inconsistent; it doesn't always lead to better performance and can even be detrimental in some cases. The authors observe that neither zero-shot nor role-play prompting is universally superior, suggesting they have complementary strengths. The challenge is to effectively combine these two approaches to leverage their respective advantages and achieve more robust and accurate reasoning.

Methods

The core of Persona Switch is a step-by-step decoding process. At each step, the method generates two candidate outputs: one using standard zero-shot prompting and another using role-play prompting. The key innovation lies in how it selects the better candidate. It compares the "output confidence" of these two candidates, measured by the logit gap. The logit gap quantifies the difference between the probability of the most likely next token and the second most likely next token. A larger logit gap indicates higher confidence. The method then selects the output with the larger logit gap for that step, and this selected output is appended to the reasoning trace. This process repeats until an end-of-sequence token is generated, effectively building the final answer by dynamically choosing the most confident path at each stage.

Datasets

The experiments were conducted on five widely-used reasoning benchmarks:

  • GSM8K: Mathematical word problems.
  • AQUA-RAT: Multiple-choice math problems.
  • CSQA: Commonsense reasoning questions.
  • Last Letter Concatenation: Symbolic reasoning task.
  • Tracking Shuffled Objects (from BIG-bench): A task involving tracking object positions through a sequence of events.

The paper does not introduce new benchmarks but uses existing ones to evaluate the proposed method.

Key Insights and Ideas

  • Complementary Strengths: Zero-shot and role-play prompting aren't mutually exclusive; they offer different, often complementary, strengths that can be combined.
  • Logit Gap as a Confidence Measure: The logit gap is a surprisingly effective and informative signal for determining the reliability of an LLM's output at a token level.
  • Stepwise Selection is Crucial: Evaluating and selecting the best output at each intermediate reasoning step (token-level or step-level) is more effective than making a single decision for the entire sequence. This allows for a more nuanced integration of different perspectives.
  • Persona-Agnostic Semantic Invariance: The authors introduce this principle, suggesting that adding a persona doesn't change the core information needed to solve a problem, but it can influence how the LLM approaches it.

Key Results

  • Performance Improvement: Persona Switch consistently outperformed competitive baselines across all five datasets, achieving up to 5.13% absolute accuracy improvement over the strongest baseline.
  • Superiority over Variants: It also demonstrated superior accuracy compared to simpler variants like "Low-gap Selection" (which always picks the lower average logit gap) and "Random Selection," highlighting the effectiveness of their specific selection mechanism.
  • Granularity Matters: Step-level selection (comparing outputs at each token generation step) yielded the best performance, outperforming token-level or full-sequence selection.
  • Qualitative Examples: The paper provides qualitative examples showing how Persona Switch can correctly answer questions where either zero-shot or role-play prompting alone fails, by integrating correct reasoning steps from both.

Limitations and Future Work

  • Fixed Prompts and Datasets: The evaluation used fixed role-play prompts and datasets with clearly defined ground-truth answers. The effectiveness of Persona Switch in more subjective or open-ended tasks where ground truth is ambiguous remains an open question.
  • Zero-Shot Reasoning Focus: The current evaluation is limited to zero-shot reasoning tasks. Future work could explore whether this stepwise selection approach is effective in other settings, such as different domains or task types beyond reasoning.
  • Persona Prompt Design: While the paper uses established prompts, exploring the impact of different persona prompt designs (e.g., verbosity, structure) could further enhance performance.

Summary updated

Abstract

Role-play prompting is known to steer the behavior of language models by injecting a persona into the prompt, improving their zero-shot reasoning capabilities. However, such improvements are inconsistent across different tasks or instances. This inconsistency suggests that zero-shot and role-play prompting may offer complementary strengths rather than one being universally superior. Building on this insight, we propose **Persona Switch**, a novel decoding method that dynamically combines the benefits of both prompting strategies. Our method proceeds step-by-step, selecting the better output between zero-shot and role-play prompting at each step by comparing their output confidence, as measured by the logit gap. Experiments with widely-used LLMs demonstrate that Persona Switch consistently outperforms competitive baselines, achieving up to 5.13% accuracy improvement. Furthermore, we show that output confidence serves as an informative measure for selecting the more reliable output.
BibTeX
@inproceedings{kim2026persona,
  author = {Junseok Kim and Nakyeong Yang and Kyomin Jung},
  title = {{Persona Switch: Mixing Distinct Perspectives in Decoding Time}},
  booktitle = {EACL (Findings)},
  year = {2026},
  url = {https://openreview.net/forum?id=LyvLNcdpxd},
}
Figure