Turning Bias into Bugs: Bandit-Guided Style Manipulation Attacks on LLM Judges

Xianglin Yang, Bryan Hooi, Gelei Deng, Tianwei Zhang, Jin Song Dong

The Vulnerability of LLM-as-a-Judge

The "LLM-as-a-judge" paradigm has rapidly become the backbone of modern AI development. Powerful models are routinely deployed to rank chatbots on leaderboards, curate training data for Reinforcement Learning from Human Feedback (RLHF), and even automate peer review for scientific papers.

However, these judges harbor well-documented stylistic biases. They often exhibit a preference for longer responses (verbosity bias), specific tones, or the inclusion of markdown and emojis—frequently prioritizing these superficial traits over factual accuracy. While the AI community has largely viewed these biases as passive flaws, new research reframes them as an active attack surface. If an adversary can systematically exploit these stylistic preferences, they can artificially inflate evaluation scores, distort leaderboards, and corrupt the datasets that future models rely on.

Introducing BITE: A Bandit-Guided Attack

To demonstrate this vulnerability, researchers developed BITE (BIas exploraTion and Exploitation), a black-box adversarial framework. BITE operates without needing access to a judge's internal weights or gradients; it only requires the final evaluation score to iteratively learn and exploit the model's preferences.

The attack is formulated as a contextual bandit problem. In each round, the attacker observes a "context" (the prompt and the current response) and selects an "action" from a curated set of eight stylistic transformations. These edits include:

  • Verbosity: Expanding the length of the answer.
  • Markdown: Injecting bold text, italics, or headers.
  • Sentiment: Shifting the tone to be more cheerful or agreeable.
  • Authority: Adding (potentially fabricated) citations or authoritative phrasing.
  • Formatting: Restructuring text into JSON or incorporating emojis.

To balance the exploration of new styles with the exploitation of proven ones, BITE employs the LinUCB algorithm. This allows the framework to dynamically learn a "vulnerability fingerprint" unique to each specific judge. The researchers also provide mathematical regret bounds, proving that BITE remains highly effective even when the judge's true scoring logic is complex and non-linear.

Experimental Setup and Key Findings

BITE was evaluated across high-stakes benchmarks, including industry-standard chatbot leaderboards (AlpacaEval 2.0, Arena-Hard-Auto) and AI peer-review tasks (MLRBench). The target judges spanned a diverse suite of state-of-the-art models, including proprietary systems like o3-mini and Gemini-2.5-Flash, as well as open-source heavyweights like Llama-3.3-70B, DeepSeek-R1, and Qwen3-235b.

Attack Efficacy

The results expose a critical weakness in automated evaluation:

  • Significant Score Inflation: BITE consistently raised evaluation scores by 1 to 2 points on a 9-point scale.
  • High Success Rate: The attack achieved a success rate exceeding 65% across various judges, significantly outperforming traditional prompt injection and jailbreaking methods (which are frequently blocked by safety filters).
  • Effectiveness on Objective Tasks: Surprisingly, BITE successfully inflated scores even on fact-based, objective questions where stylistic flourishes should theoretically be irrelevant.

Vulnerability Fingerprints and Preference Leakage

The study revealed that stylistic biases are not universal. While almost all judges favor verbosity and italics, they diverge sharply on other formatting choices. For instance, Gemini and Qwen favor the use of special characters, whereas o3-mini and DeepSeek actively penalize them. BITE's success lies in its ability to adapt to these model-specific quirks.

Furthermore, the researchers observed "preference leakage," where attack strategies proved transferable between certain models. This likely reflects a "teacher-student" dynamic: models trained on the outputs of larger, proprietary models inherit the stylistic preferences of their teachers.

The Threat of Stealth

Unlike traditional jailbreaks that rely on gibberish or obvious trigger phrases, BITE’s edits are semantically null. Embedding-based checks and secondary LLM evaluations confirmed that the modified answers maintain over 90% semantic similarity to the originals. To a human reader, the core answer remains unchanged; to the LLM judge, it appears vastly superior. Standard "style control" defenses—which attempt to mathematically neutralize the influence of length or headers—failed to stop BITE, as the algorithm discovers nuanced stylistic patterns that simple linear defenses cannot detect.

Conclusion and Future Directions

BITE successfully demonstrates that the stylistic biases of LLM judges are not just quirks, but exploitable security bugs. While the current iteration of BITE relies on a "helper LLM" to execute the stylistic rewrites (adding some computational overhead), the implications are profound.

The success of this framework highlights an urgent need for the research community to develop style-invariant judges that evaluate solely on semantic content and factual accuracy. Future work must focus on adversarial training—using BITE-generated examples to harden evaluation models—and exploring broader bias categories, such as cognitive fallacies, to secure the integrity of the LLM-as-a-judge paradigm.

Summary updated

Abstract

The known **stylistic biases** in LLM judges, such as a preference for verbosity or specific sentence structures, present an underexplored *security vulnerability*. In this work, we introduce BITE (**BI**as explora**T**ion and **E**xploitation), a black-box adversarial framework that learns semantics-preserving edits to mislead an LLM judge and *artificially* inflate the scores it assigns. We cast the selection of stylistic edits as a contextual bandit problem and use a LinUCB policy to adaptively choose edits that maximize the judge's score without access to model parameters or gradients. Empirically, we test BITE across a diverse range of LLM judges and tasks, including both pointwise and pairwise comparisons on chatbot leaderboards and AI-reviewer benchmarks. BITE achieves an attack success rate exceeding 65% and raises scores by 1–2 points on a 9-point scale, all while preserving semantic equivalence. We further assess the attack's stealthiness, showing that BITE evades standard style-control methods and several detection baselines. Our findings expose a fundamental weakness in the LLM-as-a-judge paradigm and motivate robust, attack-aware evaluation. Our code is available at https://github.com/xianglinyang/llm-as-a-judge-attack.
BibTeX
@inproceedings{yang2026turning,
  author = {XIANGLIN YANG and Bryan Hooi and Gelei Deng and Tianwei Zhang and Jin Song Dong},
  title = {{Turning Bias into Bugs: Bandit-Guided Style Manipulation Attacks on LLM Judges}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=7g23tYAIDC},
}
Figure