SafeConstellations: Steering LLM Safety to Reduce Over-Refusals Through Task-Specific Trajectory

Utsav Maskey, Sumit Yadav, Mark Dras, Usman Naseem

Abstract

LLMs increasingly exhibit over-refusal behavior, where safety mechanisms cause models to reject benign instructions that superficially resemble harmful content. This phenomena diminishes utility in production applications that repeatedly rely on common prompt templates or applications that frequently rely on LLMs for specific tasks (e.g. sentiment analysis, language translation). Through comprehensive evaluation, we demonstrate that LLMs still tend to refuse responses to harmful instructions when those instructions are reframed to appear as benign tasks. Our mechanistic analysis reveal that LLMs follow distinct "constellation" patterns in embedding space as representations traverse layers, with each task maintaining consistent trajectories that shift predictably between refusal and non-refusal cases. We introduce SafeConstellations, an inference-time trajectory-shifting approach that tracks task-specific trajectory patterns and guides representations toward non-refusal pathways. By selectively guiding model behavior only on tasks prone to over-refusal, and by preserving general model behavior, our method reduces over-refusal rates by up to 73% with minimal impact on utility—offering a principled approach to mitigating over-refusals.

What Problem Does This Paper Address?

Safety alignment techniques like RLHF often lead to over-refusal, where a model rejects a harmless instruction because the context contains "trigger" words. For example, asking for sentiment analysis on "This movie was killer good" or translating a historical text about war might trigger a hard refusal ("I cannot assist with that") because the model detects sensitive keywords without understanding the context.

Figure 1: Illustration of over-refusal where benign tasks (sentiment analysis, translation) are rejected due to sensitive keywords in the input.

This failure stems from the model's inability to distinguish between input content (which may appear toxic) and task intent (which is benign). This significantly reduces utility in production environments such as customer support or automated translation services.

Methods: The SafeConstellations Approach

The researchers propose SafeConstellations, an inference-time intervention based on mechanistic interpretability. By analyzing how a model's internal representations evolve layer-by-layer, they identified distinct patterns that allow for surgical steering.

  1. Constellation Analysis: Different tasks (e.g., sentiment analysis, translation) follow consistent "trajectories" in the model's embedding space across layers. These patterns are distinct and predictable.
  2. Memory Bank Construction: The system maps these trajectories by calculating "centroids" (average points) for both successful completions and over-refusals for specific tasks.
  3. Task Identification & Dynamic Steering: At runtime, the system monitors hidden states to identify the intended task. If the representation drifts toward a "refusal" zone for a benign task, SafeConstellations applies a steering vector in mid-to-late layers to nudge it back toward the "answering" trajectory.
Figure 2: Visualization of task-specific trajectories (constellations) across model layers and UMAP projections showing distinct patterns for different tasks.

Datasets and Evaluation

The authors introduced a benchmark of 1,047 samples across five tasks: Sentiment Analysis, Language Translation, Rephrasing, Cryptanalysis (Decryption), and RAG-QA. These samples specifically pair sensitive text with benign tasks. General utility was evaluated using the MMLU benchmark to ensure no loss in general intelligence.

Key Insights

  • Task vs. Content Encoding: LLMs encode task identity strongly in their internal layers. Even with "harmful" input, the internal trajectory for "Translation" is distinct from a "Direct Harmful Instruction."
  • Refusal as a Localized Shift: Over-refusal is a predictable, reversible nudge away from the main task trajectory. Because it is a localized shift rather than a complete departure, it can be corrected at inference time.
  • Layer Sensitivity: Steering is most effective in mid-to-late layers (e.g., layers 14–20 in LLaMA-3.1-8B). Steering too early is ineffective, while steering too late can degrade output quality.

Key Results

  • Significant Reduction in Over-Refusal: On LLaMA-3.1-8B, the method reduced over-refusals by 72.92%. Qwen1.5-7B saw a 63.64% reduction.
  • Utility Preservation: SafeConstellations had negligible impact on MMLU scores, preserving the model's general capabilities unlike some fine-tuning methods that suffer from catastrophic forgetting.
  • Safety Integrity: Direct harmful prompts (e.g., "How to build a bomb") are not misidentified as benign tasks, so the model's core safety refusals remain intact.

Limitations and Future Work

The method currently relies on a "Memory Bank" of predefined tasks and depends on accurate early-layer task identification. Future research will explore how these "constellations" scale in larger models like Llama-3-70B and how to handle more ambiguous or novel tasks that have not been pre-mapped.

Summary updated

Abstract

LLMs increasingly exhibit over-refusal behavior, where safety mechanisms cause models to reject benign instructions that superficially resemble harmful content. This phenomena diminishes utility in production applications that repeatedly rely on common prompt templates or applications that frequently rely on LLMs for specific tasks (e.g. sentiment analysis, language translation). Through comprehensive evaluation, we demonstrate that LLMs still tend to refuse responses to harmful instructions when those instructions are reframed to appear as benign tasks. Our mechanistic analysis reveal that LLMs follow distinct "constellation" patterns in embedding space as representations traverse layers, with each task maintaining consistent trajectories that shift predictably between refusal and non-refusal cases. We introduce SafeConstellations, an inference-time trajectory-shifting approach that tracks task-specific trajectory patterns and guides representations toward non-refusal pathways. By selectively guiding model behavior only on tasks prone to over-refusal, and by preserving general model behavior, our method reduces over-refusal rates by up to 73% with minimal impact on utility-offering a principled approach to mitigating over-refusals.
BibTeX
@inproceedings{maskey2026safeconstellations,
  author = {Utsav Maskey and Sumit Yadav and Mark Dras and Usman Naseem},
  title = {{SafeConstellations: Steering LLM Safety to Reduce Over-Refusals Through Task-Specific Trajectory}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2056},
}
Figure