Which Words a Model Fears: From a Refusal Direction to a Vocabulary-Wide Fear Lexicon

Boyuan Zhang, Mohammad Zandsalimy, Shanu SushmitaarXiv 2026 · arXiv

TL;DR

Aligned LLMs often refuse safe prompts not because of the request's intent, but due to specific "trigger" words. This paper identifies a causal "refusal direction" in activation space and uses it to build a vocabulary-wide "fear lexicon," demonstrating that swapping a "feared" word for a "courageous" synonym (like "poetic" or "whimsical") can significantly mitigate over-refusal.

What Problem Does This Paper Address?

Safety alignment in Large Language Models (LLMs) is a double-edged sword. While it prevents harmful outputs, it often leads to over-refusal: the model declines perfectly benign requests (e.g., "How do I kill a Python process?") because they contain "unsafe-looking" cues.

Previous research suggested that surface-level words act as triggers, but we lacked a way to quantify how much each word matters or a systematic method to fix it without retraining. The authors set out to measure word-level sensitivity across the entire vocabulary and determine if this sensitivity is mediated by a specific, locatable direction in the model's internal representations.

Methods

The researchers employed a multi-step approach across eight instruction-tuned models (including Qwen3.5, Llama-3.2, and Gemma-4):

  1. Crossed Differencing & ANOVA: To isolate the effect of words from the effect of the prompt, they used a "crossed design." They appended pairs of harm/neutral words (e.g., bomb vs. garden) to various prompts and measured the difference in hidden states at the generation boundary (the last tokens before the model starts generating an answer). They used a vector ANOVA to extract a shared "refusal direction" ($\mu$).
  2. Activation Steering: To prove causality, they "wrote" along this direction—adding or subtracting the vector from the model's activations during inference.
  3. Vocabulary Scanning: Using the refusal direction as a ruler, they projected the activations of over 71,000 words to create a "fear table" (lexicon). Words that pushed activations strongly along the refusal direction were labeled "feared," while those pushing the opposite way were labeled "courageous."
  4. Table-Guided Substitution: They tested if the lexicon could predict behavior by replacing the most "feared" word in a prompt with its most "courageous" meaning-preserving synonym.

Datasets

The study utilized several established benchmarks to test different aspects of refusal:

  • OR-Bench (Hard-1K and Toxic): Used for measuring over-refusal on benign prompts and standard refusal on harmful ones.
  • XSTest: Provided matched safe and contrast prompts to identify exaggerated safety behaviors.
  • WildJailbreak: Used to source a pool of harmful prompts.
  • StrongREJECT & IFEval: Used as secondary metrics to ensure that reducing refusal didn't inadvertently increase harmfulness or break instruction-following capabilities.

Key Insights and Ideas

  • Refusal is Added Late: Mechanistic analysis on Qwen3.5-4B revealed that while a "harm" word is recognized early, its effect on refusal is primarily written by late MLP layers at the generation boundary.
  • Intuition is a Poor Guide: The authors found that "courage" words aren't just generic safe terms like "educational" or "professional." Instead, stylistic "register" words like poetic, whimsical, and sardonic were the most effective at lowering refusal.
  • Predictable "Fear": A word's position in the fear lexicon is highly predictable ($ \rho \approx 0.84 $) from its context-free input embedding. This suggests models learn to associate specific regions of the embedding space with safety triggers.
  • Shared vs. Specific: While models share a "common core" of feared words (slurs, violent crime terms), each model also has unique lexical sensitivities.

Key Results

  • Causal Steering: Writing the refusal direction at a dose of -1.5 removed between 41% and 92% of the refusal caused by inserting a harm word across all eight models.
  • The Fear Lexicon: The vocabulary scan produced a clear ranking. At the "feared" end were sexual terms and slurs; at the "courageous" end were whimsical and poetic terms.
The Qwen3.5-4B fear table showing the distribution of the entire vocabulary. Feared words (red) sit in the positive tail, while courage words (poetic, whimsical) form the negative tail.
  • Behavioral Prediction: In held-out tests, replacing the most feared word in a benign prompt with a courageous synonym lowered refusal rates by 3 to 28 percentage points.
  • Insertion Impact: Simply appending a harm word to a prompt the model would otherwise answer raised refusal rates by up to 49 points.
Figure 2: One inserted harm word raises Qwen3.5-4B refusal at every position. Each panel mirrors two his- tograms of the per-prompt refusal rate over ten samples on one vertical axis: the control prompt (gray, left) against the same prompt with the word inserted (blue, right), over the prompts passing that position’s detection gate. Dashed lines mark the means; the arrow is the paired mean increase with its 95% bootstrap interval, above the share of prompts whose rate rose or fell. The orange strip is the share of the 321 prompts the gate admits (55, 105, 267). Ungated, over all 321 prompts: +0.32, +0.18, +0.30.

Limitations and Future Work

  • Model Scale: The study focused on open-weight models up to 27B parameters; it remains to be seen if these findings hold for massive closed models like GPT-4 or Claude 3.
  • Language Scope: The current lexicon and analysis are limited to English.
  • Dual-Use Risk: The authors acknowledge that the same techniques used to reduce over-refusal could potentially be used to bypass legitimate safety filters, though the effect on harmful prompts was modest (+0.04 rise in harmfulness).
  • Future Direction: The authors suggest that safety evaluation should control for vocabulary to be more reliable and that the fear table can serve as a diagnostic tool for developers to audit model oversensitivity.

Summary updated

Abstract

Whether an aligned language model refuses depends in part on which words a prompt contains, regardless of whether those words carry the request's meaning. Harm words shift the hidden state at the generation boundary along a component shared across prompts and words, which crossed differencing extracts as a single direction. In eight models, writing along this direction raises and lowers refusal monotonically with dose, ahead of random and KL-matched directions. Used as a ruler over the full vocabulary, the direction gives every word a shift, from "feared" words at one end to "courage" words at the other; a word's shift is predictable from its own representation, and appending a feared word to a benign prompt is enough to make the model refuse it. Two held-out tests validate the ranking in all eight models: refusal after inserting an unseen word rises with its table shift, and replacing a prompt's most feared word by its most courageous meaning-preserving synonym significantly raises the answer rate on benign and harmful prompts alike, by an amount that follows the shift the replacement actually causes.
BibTeX
@misc{zhang2026which,
  author = {Boyuan Zhang and Mohammad Zandsalimy and Shanu Sushmita},
  title = {{Which Words a Model Fears: From a Refusal Direction to a Vocabulary-Wide Fear Lexicon}},
  year = {2026},
  eprint = {2611.02348},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.02348},
}
Figure