NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning

Xin Yi, Shunfan Zheng, Linlin Wang, Gerard de Melo, Xiaoling Wang, Liang He

Abstract

Large Language Models (LLMs) are increasingly deployed via "fine-tuning-as-a-service" (FTaaS), allowing users to customize models for specific tasks. However, this flexibility introduces a significant security risk: "harmful fine-tuning." Even a tiny fraction of malicious data (as low as 1%) can cause a model's safety guardrails to collapse. We introduce Neuron-Level Safety Realignment (NLSR), a training-free framework designed to surgically restore safety. NLSR identifies safety-critical neurons and replaces compromised ones with healthy versions from a super-aligned reference model. Experimental results demonstrate that NLSR significantly reduces harmfulness while maintaining near-perfect utility on downstream tasks.

Introduction

The widespread adoption of LLMs has necessitated robust safety alignment, typically achieved through Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF). Despite these efforts, the rise of FTaaS has created a loophole. When users fine-tune an aligned model on custom datasets, they may inadvertently or intentionally trigger "alignment drift," where the model's safety constraints are weakened.

Overview of the NLSR framework showing how it restores safety in customized LLMs.

Existing defenses against such attacks often involve retraining the model or applying broad, layer-wise updates. These methods are computationally expensive and frequently degrade the model's performance on the target task. NLSR addresses these challenges by offering a surgical, training-free approach that operates at the individual neuron level.

Methodology: The NLSR Framework

The NLSR framework consists of three primary stages designed to identify and fix safety-broken components without requiring additional gradient steps.

Detailed architectural workflow of the three stages in NLSR.

1. Construction of a Safety Reference Model

To effectively identify safety-critical neurons, the framework first constructs a "super-aligned" reference model ($F_{w_e}$). This is achieved through LoRA extrapolation, which interpolates between a weakly aligned model and a strongly aligned one. This process amplifies safety-related features, making the "safety signal" within the weights more discernible.

2. Recognition of Safety-Critical Neurons

Once the reference model is established, NLSR identifies the specific neurons responsible for safety. By applying Singular Value Decomposition (SVD) and rank reduction to the LoRA weights, the system generates a "position mask" ($M$) that highlights safety-critical components. This allows the system to focus its restoration efforts only on the neurons that matter most for alignment.

3. Neuron-Level Restoration

The final stage involves a "transplant" of healthy neurons into the customized model ($F_{W_t}$). This stage includes:

  • Similarity Check: Comparing the safety-critical neurons of the customized model against the super-aligned reference.
  • Adaptive Layer Pruning: A probability-based selection process determines which layers require intervention. Layers that retain high similarity to the reference are left untouched to preserve task-specific utility.
  • Neuron-Level Correction: For compromised layers, the "broken" neurons are replaced with "patches" derived from the reference model. This mathematical update to the LoRA weights is entirely training-free.

Experimental Evaluation

The authors evaluated NLSR using benchmarks such as PKU-SafeRLHF and BeaverTails for safety, and SST-2, AGNEWS, and GSM8K for utility.

Key Results

  • Safety Restoration: In sentiment analysis tasks (SST-2), NLSR reduced the harmfulness score from 61.1% (in a standard aligned model) to 22.8%. This outperformed the previous state-of-the-art method, SafeLoRA, which achieved a score of 53.1%.
  • Utility Preservation: Remarkably, NLSR maintained a fine-tuning accuracy of 95.1% on SST-2, showing virtually no performance loss compared to the compromised model.
  • Robustness and Efficiency: The framework remained effective even when the poisoning ratio of the fine-tuning data reached 30%. Furthermore, because it is training-free, the realignment process occurs in seconds, making it highly suitable for real-time deployment in FTaaS pipelines.

Discussion and Conclusion

NLSR demonstrates that safety in LLMs is "regional" and can be managed with surgical precision. By focusing on individual neurons rather than entire layers, the framework avoids the "blunt instrument" problem of previous defenses.

While currently optimized for LoRA-based fine-tuning, future work could extend these principles to full-parameter updates. NLSR represents a significant step toward practical, efficient, and robust safety guardrails for the next generation of customizable AI models.

Summary updated

Abstract

The emergence of fine-tuning-as-a-service has revealed a new vulnerability in large language models (LLMs). A mere handful of malicious data uploaded by users can subtly manipulate the fine-tuning process, leading to a compromised alignment state. Existing methods to counteract fine-tuning attacks typically require substantial computational resources. Even with parameter-efficient techniques like LoRA, gradient updates remain essential. To address these challenges, we propose Neuron-Level Safety Realignment (NLSR), a training-free framework that restores the safety of LLMs based on the similarity difference of safety-critical neurons before and after fine-tuning. The core of our framework is first to construct a safety reference model from an initially aligned model to amplify safety-related features in neurons. We then utilize this reference model to identify safety-critical neurons, which we prepare as patches. Finally, we selectively restore only those neurons that exhibit significant similarity differences by transplanting these prepared patches, thereby minimally altering the fine-tuned model. Extensive experiments demonstrate significant safety enhancements in fine-tuned models across multiple downstream tasks, while greatly maintaining task-level accuracy. Our findings indicate that safety-critical neurons exhibit significant regional variations after fine-tuning, which can be effectively corrected through neuron transplantation from the reference model without the need for additional training.
BibTeX
@inproceedings{yi2025nlsr,
  author = {Xin Yi and Shunfan Zheng and Linlin Wang and Gerard de Melo and Xiaoling Wang and Liang He},
  title = {{NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i24.34762},
}
Figure