Sparse Activation Editing for Reliable Instruction Following in Narratives

Runcong Zhao, Chengyu Cao, Qinglin Zhu, Xiucheng Lv, Shun Shao, Lin Gui, Ruifeng Xu, Yulan He

Abstract

Large Language Models (LLMs) increasingly serve as the backbone for interactive narrative agents and role-playing applications. However, these models often struggle with a "tug-of-war" between global narrative constraints (e.g., maintaining a specific genre or setting) and immediate user prompts that may contradict those rules. This conflict typically results in two failure modes: "No Control," where the model breaks narrative logic to follow the user, or "Over Control," where the model flatly refuses to engage, breaking immersion. We present Concise-SAE, a training-free framework that utilizes Sparse Autoencoders (SAEs) to surgically edit model activations. By identifying monosemantic features and employing Bayesian Optimization to balance bidirectional steering, Concise-SAE achieves up to a 3x improvement in instruction following across multiple model families while preserving linguistic fluency.

Introduction

In narrative-rich environments, LLMs must navigate complex, multi-layered instructions. A realistic detective story requires the model to adhere to the laws of physics and logic, even when a user attempts to introduce supernatural elements. Current methods for steering model behavior, such as representation engineering, often rely on expensive, manually labeled contrastive datasets. Furthermore, these methods frequently treat instruction adherence as a linear spectrum, assuming that "following" and "violating" an instruction are simple opposites.

Concise-SAE addresses these limitations by treating instruction adherence as a multi-dimensional problem. By leveraging the interpretability of Sparse Autoencoders, the framework identifies specific internal "features" that correspond to narrative constraints and user-driven violations, allowing for precise, automated steering without the need for fine-tuning.

The Concise-SAE Framework

The Concise-SAE pipeline consists of three primary stages designed to locate, target, and optimize internal model representations.

1. Localisation via Semantic Aggregation

To identify relevant internal features without labeled data, the framework prompts the LLM to generate its own examples of adherence and violation. A novel "semantic aggregation" technique is then applied: a specific keyword (e.g., "realistic") is appended to the input. Due to the attention mechanism in Transformers, this final token naturally aggregates the preceding context. This aggregated vector is passed through a Sparse Autoencoder (SAE) to isolate monosemantic features—individual dimensions that represent discrete concepts—thereby denoising the signal more effectively than standard token-averaging methods.

2. Bidirectional Steering

A key insight of this research is the "orthogonality" of behavioral features. The authors discovered that the features representing "following an instruction" and "violating an instruction" are often independent rather than polar opposites. Consequently, Concise-SAE employs bidirectional steering: it simultaneously amplifies the top $k$ features that support the narrative constraints and suppresses the top $k$ features that lead to violations. This dual approach ensures the model is guided toward the desired behavior rather than just being pushed away from the undesired one.

3. Bayesian Optimization (BO)

Determining the optimal "strength" of an internal edit is a significant challenge in representation engineering. Excessive steering can lead to incoherence or repetitive outputs, while insufficient steering fails to change behavior. Concise-SAE utilizes Bayesian Optimization to automatically discover the ideal coefficients for the steering vector. The BO process evaluates candidate edits using a reward function that balances three critical factors: instruction compliance, refusal avoidance, and overall output quality.

Experimental Evaluation

Datasets and Benchmarks

The researchers introduced FREEINSTRUCT, a benchmark of 1,212 examples specifically designed for narrative-rich, adversarial settings. Each entry pairs a story context with both a standard user input and an adversarial input intended to trigger a rule violation. To test the generalizability of the method, the authors also evaluated Concise-SAE on:

  • WildGuard: For adversarial prompt detection.
  • Prompt Injection: Testing resistance to attacks that attempt to override system instructions.
  • XSTest: To monitor for "over-refusal," ensuring the model remains helpful for benign queries.

Results

Concise-SAE demonstrated significant performance gains across several state-of-the-art models, including Gemma-2 (2B and 9B) and Llama-3.1 (8B). On the FREEINSTRUCT dataset, the method achieved a relative improvement in the Instruction Following Rate (IFR) of 2.3x for Gemma-2-2B and nearly 3x for Gemma-2-9B compared to the baseline.

Crucially, the method maintained high Output Quality (OQ) scores. Unlike many steering techniques that degrade fluency, the automated balancing provided by Bayesian Optimization allowed Concise-SAE to outperform traditional baselines like In-Context Learning (ICL) and In-Context Vectors (ICV) in terms of response coherence.

Discussion and Insights

The "Goldilocks" Zone of Editing

The study highlights the existence of a "sweet spot" for internal model edits. The use of SAEs allows for a more surgical intervention than traditional principal component analysis (PCA) methods, but the strength of the intervention remains the deciding factor for success. The success of the BO step suggests that automated hyperparameter tuning is essential for making representation engineering practical for real-world applications.

Generalization to Safety

Interestingly, the improvements observed in narrative steering also translated to better performance on safety benchmarks like WildGuard. This suggests that the underlying mechanisms for following narrative rules and adhering to safety guidelines are related, potentially allowing for a unified approach to model alignment through activation editing.

Conclusion

Concise-SAE represents a significant step forward in the precise control of LLM behavior. By combining the interpretability of Sparse Autoencoders with the efficiency of Bayesian Optimization, the framework provides a training-free solution to the persistent problem of instruction adherence in complex contexts. Future work may explore the application of these surgical edits to multi-step reasoning tasks and the development of even more efficient optimization techniques to reduce the latency associated with the self-evaluation phase.

Summary updated

BibTeX
@misc{zhao2025sparse,
  author = {Runcong Zhao and Chengyu Cao and Qinglin Zhu and Xiucheng Lv and Shun Shao and Lin Gui and Ruifeng Xu and Yulan He},
  title = {{Sparse Activation Editing for Reliable Instruction Following in Narratives}},
  year = {2025},
  eprint = {2505.16505},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.16505},
}
Figure