NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning

Xin Yi, Shunfan Zheng, Linlin Wang, Gerard de Melo, Xiaoling Wang, Liang He

Abstract

The emergence of finetuning-as-a-service has revealed a new vulnerability in large language models (LLMs). A mere handful of malicious data uploaded by users can subtly manipulate the finetuning process, resulting in an alignment-broken model. Existing methods to counteract fine-tuning attacks typically require substantial computational resources. Even with parameter-efficient techniques like LoRA, gradient updates remain essential. To address these challenges, we propose Neuron-Level Safety Realignment (NLSR), a training-free framework that restores the safety of LLMs based on the similarity difference of safety-critical neurons before and after fine-tuning. The core of our framework is first to construct a safety reference model from an initially aligned model to amplify safety-related features in neurons. We then utilize this reference model to identify safety-critical neurons, which we prepare as patches. Finally, we selectively restore only those neurons that exhibit significant similarity differences by transplanting these prepared patches, thereby minimally altering the fine-tuned model. Extensive experiments demonstrate significant safety enhancements in fine-tuned models across multiple downstream tasks, while greatly maintaining task-level accuracy.

Introduction

The widespread adoption of "fine-tuning-as-a-service" has introduced significant security risks. While base models are typically aligned to ensure safety, research indicates that fine-tuning on even a minimal amount of malicious data (as little as 1%) can compromise these safeguards, leading to "harmful fine-tuning attacks." Existing defenses often necessitate expensive retraining or impose constraints that hinder the model's ability to learn new tasks.

To bridge this gap, we introduce Neuron-Level Safety Realignment (NLSR). NLSR is a training-free framework designed to restore safety in compromised LLMs by surgically "patching" specific safety-critical neurons. By leveraging a super-aligned reference model, NLSR reduces harmful outputs while preserving performance on downstream tasks.

Methodology

The NLSR framework operates in three primary stages, as illustrated in Figure 2.

Overview of the NLSR framework, showing the construction of a safety reference model, recognition of safety-critical neurons, and neuron-level restoration.
  1. Construction of a Safety Reference Model: Instead of relying solely on the original aligned model, NLSR employs "weak-to-strong extrapolation" to create a "super-aligned" reference. This involves amplifying safety-related weights within a LoRA (Low-Rank Adaptation) configuration to make safety-critical features more discernible.
  2. Recognition of Safety-Critical Neurons: Utilizing Singular Value Decomposition (SVD), the framework identifies neurons most responsible for safety behaviors, creating a spatial map of safety logic within the model's layers.
  3. Neuron-Level Correction: The system compares safety neurons of the fine-tuned model with those of the reference. If similarity is low—indicating corruption—healthy neurons are "transplanted" from the reference. This process uses probability-based layer pruning to ensure only compromised components are corrected.

Experimental Results and Insights

The effectiveness of NLSR was evaluated across benchmarks including SST-2, AGNEWS, and GSM8K, with safety measured via HarmBench and QA-Moderation.

A key finding is that harmful fine-tuning disrupts output patterns rather than erasing safety concepts. Hidden state analysis showed that "safe" vs. "unsafe" inputs could still be classified with over 95% accuracy. NLSR's surgical approach targets individual neurons, preserving task-specific knowledge while reverting only the parameters causing safety leaks.

Impact of the pre-amplification coefficient beta on model accuracy across various benchmarks and the corresponding harmfulness score on BeaverTails.

As shown in Figure 5, the pre-amplification coefficient $\beta$ plays a crucial role. Increasing $\beta$ significantly reduces the harmfulness score (measured on BeaverTails) while maintaining or even slightly improving accuracy on downstream tasks like tinyMMLU and tinyHellaswag. This demonstrates that the "super-aligned" reference provides a clearer signal for identifying and restoring safety-critical neurons without degrading general performance.

Quantitatively, NLSR reduced the Harmfulness Score by 38.3% on SST-2 compared to an attacked model. It outperformed state-of-the-art methods like SafeLoRA, achieving a 30.3% lower harmfulness score and a 1.1% gain in task accuracy. The method proved robust across architectures including Llama3-8B, Qwen2-7B, and Mistral-7B.

Conclusion

NLSR offers a precise, training-free solution for realigning LLMs after harmful fine-tuning. By focusing on neuron-level restoration, it maintains a balance between safety and utility. Future work will address dependencies on initial alignment quality, scalability to full-parameter fine-tuning, and resilience against adaptive attacks.

Summary updated

Abstract

The emergence of finetuning-as-a-service has revealed a new vulnerability in large language models (LLMs). A mere handful of malicious data uploaded by users can subtly manipulate the finetuning process, resulting in an alignment-broken model. Existing methods to counteract fine-tuning attacks typically require substantial computational resources. Even with parameter-efficient techniques like LoRA, gradient updates remain essential. To address these challenges, we propose \textbf{N}euron-\textbf{L}evel \textbf{S}afety \textbf{R}ealignment (\textbf{NLSR}), a training-free framework that restores the safety of LLMs based on the similarity difference of safety-critical neurons before and after fine-tuning. The core of our framework is first to construct a safety reference model from an initially aligned model to amplify safety-related features in neurons. We then utilize this reference model to identify safety-critical neurons, which we prepare as patches. Finally, we selectively restore only those neurons that exhibit significant similarity differences by transplanting these prepared patches, thereby minimally altering the fine-tuned model. Extensive experiments demonstrate significant safety enhancements in fine-tuned models across multiple downstream tasks, while greatly maintaining task-level accuracy. Our findings suggest regions of some safety-critical neurons show noticeable differences after fine-tuning, which can be effectively corrected by transplanting neurons from the reference model without requiring additional training. The code will be available at \url{https://github.com/xinykou/NLSR}
BibTeX
@inproceedings{yi2025nlsr,
  author = {Xin Yi and Shunfan Zheng and Linlin Wang and Gerard de Melo and Xiaoling Wang and Liang He},
  title = {{NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i24.34762},
}
Figure