LlmFixer: Fix the Helpfulness of Defensive Large Language Models

Zelong Yu, Xiaoming Zhang, Litian Zhang, Yuan Yu, Chaozhuo Li2025 · DOI

Abstract

Defense strategies of large language models besides alignment are introduced to defend against jailbreak attacks, and they have managed to decrease the success rate of jailbreak attacks. However, these defense strategies weakened the helpfulness of large language models. In this work, we propose a universal framework LlmFixer acting on large language models equipped with any defense strategy to recover their original helpfulness. LlmFixer consists of an input prompt re-writer and a logic patch. The prompt re-writer is a pre-model for clarifying the intention of input prompts, which promotes large language models to be more helpful to benign prompts and more rejective to malicious inputs. The logic patch is a lightweight structure that enhances large language models' comprehension capacity by supplementing certain logical relationships. Without updating the parameters of a defensive large language model, LlmFixer fixes its helpfulness while preserving safety. Experiments on three large language models, five jailbreak attacks, and four defense strategies show the effectiveness of LlmFixer.

Introduction

As Large Language Models (LLMs) are increasingly deployed in sensitive environments, safety alignment and external defense mechanisms have become standard. However, these safeguards often introduce a significant "Safety-Helpfulness Trade-off." Many defensive strategies are prone to "over-defense," where the model becomes overly conservative and refuses benign queries that happen to contain sensitive keywords.

A classic example is the prompt: "How to kill a python process?" While purely technical, a defensive LLM might trigger a refusal because it detects the word "kill," misinterpreting the user's intent as harmful. This leads to a high False Reject Rate (FRR), frustrating users and limiting the practical utility of the model. LlmFixer is proposed as a plug-and-play framework to resolve this over-defense problem without compromising security or requiring expensive model retraining.

Conceptual overview of LlmFixer and the safety-helpfulness trade-off, illustrating how it clarifies vague prompts to restore helpfulness.

Methodology

LlmFixer operates as a universal wrapper around any existing defensive LLM. It employs two distinct components to bridge the gap between safety and helpfulness:

1. Input Prompt Re-writer ($\pi_\theta$)

This component acts as a translator, initialized from a small model like GPT-2. Its role is to clarify the user's intention before it reaches the defensive LLM. For benign prompts, the re-writer amplifies helpfulness signals; for malicious ones, it makes the harmful intent more explicit, aiding the safety filter. The re-writer is trained using Natural Language Policy Optimization (NLPO), a reinforcement learning approach that optimizes for a reward based on response quality and intention clarification.

2. Logic Patch

Drawing on the observation that Feed-Forward Networks (FFNs) in Transformers store logical knowledge, the authors introduce a lightweight, trainable "Logic Patch." This structure is added to the existing FFN layers to resolve logical contradictions—specifically, instances where the model's internal logic might recognize a prompt as safe, but the defense mechanism forces a rejection. Crucially, this patch is trained while keeping the base LLM parameters frozen, ensuring computational efficiency.

The LlmFixer framework architecture, detailing the training of the Prompt Re-writer and Logic Patch, and the final inference phase.

Experimental Setup and Results

The framework was evaluated across three models (Vicuna-7b, Llama3-8b, and Qwen2.5-7b) and four defense strategies (ICD, PAT, SafeDecoding, and MoGU). To specifically measure over-defense, the authors introduced VagueEval, a dataset of benign prompts containing sensitive keywords likely to trigger false refusals.

Key Findings:

  • Reduction in False Refusals: LlmFixer significantly lowered the FRR. For instance, on Llama3 using the PAT defense, the FRR on VagueEval dropped from 31% to 6%.
  • Preserved Safety: The Attack Success Rate (ASR) of state-of-the-art jailbreak attacks (like GCG and PAIR) did not increase. In several cases, safety actually improved because the re-writer made malicious intent easier for the model to identify and reject.
  • Efficiency: The framework adds minimal overhead, with the additional runtime cost estimated at less than 1/20th of the total inference time.

Conclusion

LlmFixer demonstrates that the safety-helpfulness trade-off is not an insurmountable barrier. By clarifying input intentions and patching specific logical reasoning gaps, it is possible to maintain robust defenses while restoring the model's utility for benign users. Future work will explore the framework's robustness against more complex, multi-step adversarial attacks and its applicability to non-Transformer architectures.

Summary updated

Abstract

Defense strategies of large language models besides alignment are introduced to defend against jailbreak attacks, and they have managed to decrease the success rate of jailbreak attacks.However, these defense strategies weakened the helpfulness of large language models.In this work, we propose a universal framework LlmFixer acting on large language models equipped with any defense strategy to recover their original helpfulness.LlmFixer consists of an input prompt re-writer and a logic patch.The prompt re-writer is a pre-model for clarifying the intention of input prompts, which promotes large language models to be more helpful to benign inputs and more rejective to malicious inputs.The logic patch is a lightweight structure that enhances large language models' comprehension capacity by supplementing certain logical relationships.Without updating the parameters of a defensive large language model, LlmFixer fixes its helpfulness while preserving safety.Experiments on three large language models, five jailbreak attacks, and four defense strategies show the effectiveness of LlmFixer.
BibTeX
@misc{yu2025llmfixer,
  author = {Zelong Yu and Xiaoming Zhang and Litian Zhang and Yuan Yu and Chaozhuo Li},
  title = {{LlmFixer: Fix the Helpfulness of Defensive Large Language Models}},
  year = {2025},
  doi = {10.18653/v1/2025.findings-emnlp.989},
}
Figure