Safety Alignment Degradation in Long Contexts

21 concepts7 papers1 proposal

Modern Large Language Models (LLMs) routinely process context windows spanning tens of thousands or even millions of tokens. While their capability to retrieve information across these vast horizons has scaled dramatically, their safety guardrails have not kept pace. A striking "safety misalignment" emerges: models that rigorously reject harmful prompts in short-form interactions frequently fail when those same prompts are embedded within extensive text.

As evaluated by Lu et al. in their comprehensive benchmark LongSafety: Evaluating Long-Context Safety of Large Language Models, top-tier models experience catastrophic drops in safety compliance—often plummeting below 55%—when processing long-context inputs. Understanding why this degradation occurs requires looking past simple data volume; the failure is rooted in a fundamental mismatch between how models are aligned and how attention mechanisms operate over long sequences.

Position-Blindness and Signal Dilution

The primary driver of long-context vulnerability is that standard safety training techniques, such as Reinforcement Learning from Human Feedback (RLHF), are fundamentally position-blind. They train the model to enforce behavioral constraints on short inputs but do not explicitly teach the model to maintain these constraints as the sequence length grows.

When a model processes a massive context window, the attention weights allocated to safety instructions are effectively diluted. Kim et al. demonstrated this phenomenon in What Really Matters in Many-Shot Attacks? An Empirical Study of Long-Context Vulnerabilities in LLMs. They discovered that successful "Many-Shot Jailbreaking" does not even require malicious examples; simply filling the context window with repetitive benign text or random "Lorem Ipsum" placeholders is sufficient to bypass safety filters. Because the alignment is position-blind, the sheer distance between the initial safety constraints and the final target query causes the model's defensive state to decay, diluting the safety constraints until they are ignored.

Out-of-Distribution Task Environments

Another core reason for this failure is a severe distributional shift in training data. Standard safety alignment datasets are overwhelmingly composed of short-form question-and-answering (QA) pairs. When an aligned model is deployed in a long-context scenario—such as Retrieval-Augmented Generation (RAG) or document summarization—it enters an out-of-distribution (OOD) environment.

As Leng et al. note in Long Context RAG Performance of Large Language Models, the lack of long-context safety data during training leaves models unprepared for unique failure modes at extended lengths. This vulnerability is easily exploited in downstream NLP tasks. For instance, Liu et al. showed in Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attack that safety filters fail to recognize harmful intent when it is embedded within complex task structures like long-form summarization. Because the model is optimized to execute the task over a massive document, the embedded harmful context bypasses the short-form safety filters.

Exploiting the Haystack and Shallow Alignment

Adversaries can exploit these structural blind spots through targeted spatial and temporal positioning. In Jailbreaking in the Haystack, Shah et al. introduced the NINJA (Needle-In-haystack Jailbreak Attack) technique. They found that placing a harmful goal at the very beginning of a massive "haystack" of benign, model-generated text significantly increases attack success. The vast volume of subsequent text effectively masks the malicious intent from the model's active safety boundaries.

A parallel vulnerability exists on the output generation side, known as shallow alignment. Guardrails are often highly effective at the initial tokens of a response but diminish as the generation continues. In AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs, Lv et al. demonstrated that pre-filling a model's output with safe or affirmative text tricks the model into a state of "pseudo-termination." Once the initial tokens bypass the front-loaded safety filters, the model's defensive posture drops, allowing it to transition into generating harmful content later in the sequence.

The Path Forward: Position-Aware Alignment

The recurring evidence across these studies proves that long-context safety cannot be solved by a naive combination of existing techniques. As argued by Huang et al. in LongSafety: Enhance Safety for Long-Context LLMs, simply mixing short-context safety data with general long-context pre-training fails to create robust defenses.

To close this alignment generalization gap, training methodologies must shift toward position-aware safety alignment. This involves:

  1. Specialized Fine-Tuning Datasets: Utilizing datasets that explicitly simulate long-context attack vectors (such as those introduced by Huang et al.) to expose the model to harmful intent buried inside massive contexts during training.
  2. Persistent Defensive Constraints: Developing training objectives that penalize safety degradation across the entire span of the context window, ensuring that defensive constraints remain persistent regardless of token distance or output length.

Key Foundational Work

LongSafety: Enhance Safety for Long-Context LLMs (2025) — contributed 6 concept elements to this direction.
What Really Matters in Many-Shot Attacks? An Empirical Study of Long-Context Vulnerabilities in LLMs (2025) — contributed 4 concept elements to this direction.
AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs (2024) — contributed 3 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_long_context_safety_misalignment_565f7a7e,
  title = {long-context safety misalignment},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/long-context-safety-misalignment/rdr_565f7a7e91334cdfac455a6ca2378001},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_long_context_safety_misalignment_565f7a7e_v1,
  title = {long-context safety misalignment},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/long-context-safety-misalignment/rdr_565f7a7e91334cdfac455a6ca2378001/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake