Introduction
Large Language Models (LLMs) have become central to modern software applications, yet they face a critical security vulnerability: prompt injection. Currently ranked as the top security risk for LLM applications, prompt injection occurs when a malicious user embeds instructions within untrusted input data to override the developer's original system prompt. This vulnerability stems from the "flat" nature of traditional LLM processing, where control instructions and user data are concatenated into a single string. Consequently, the model fails to distinguish between the control plane (developer commands) and the data plane (user input).
The StruQ Framework
To address this fundamental design flaw, we introduce StruQ, a defense mechanism inspired by parameterized queries in SQL. StruQ enforces a strict separation between instructions and data through a two-component system: a secure front-end and a structured instruction-tuned model.
1. Secure Front-End and Recursive Filtering
The first line of defense is a secure front-end that transforms raw inputs into a structured format. Instead of using standard text delimiters (e.g., "### Instruction:"), which are easily spoofed, StruQ wraps inputs in special reserved tokens (e.g., [INST], [INPT], [RESP]). These tokens are added to the model's vocabulary and are inaccessible to standard user input.
To prevent "escape" attacks, the front-end employs a recursive data filter. This algorithm strips any attempts by a user to manually type reserved tokens or use control characters (like backspaces or carriage returns) to manipulate the model's internal state.
2. Structured Instruction Tuning
The core innovation of StruQ is Structured Instruction Tuning. Simply changing the input format is insufficient if the model still prioritizes any instruction it sees. We fine-tuned base models (such as Llama-7B and Mistral-7B) using a specialized dataset composition:
- Clean Samples (50%): Standard instruction-following data to maintain general utility.
- Attacked Samples (50%): Data containing "fake" instructions injected into the
[INPT]block. During training, the model is penalized for following these injected commands and rewarded for adhering strictly to the developer's original prompt in the[INST]block.
This "negative training" teaches the LLM a crucial semantic rule: text contained within the data channel must be treated as passive information, never as executable commands.
Evaluation and Results
We evaluated StruQ across three dimensions: manual injection attacks, complex automated attacks, and model utility.
Security Performance
StruQ demonstrates remarkable resilience against a wide array of attacks. For manual prompt injections—the most common threat—the success rate dropped from near-certainty in undefended models to 0% in StruQ-defended versions.
Against more sophisticated threats, such as "Completion" attacks (where attackers simulate a model response to terminate the current task), StruQ maintained a success rate below 2%. Even against state-of-the-art optimization-based attacks like TAP (Tree-of-Attacks with Pruning), Llama's vulnerability plummeted from 97% to 9%. While white-box gradient attacks (GCG) remain a challenge, StruQ still reduced the success rate from 97% to 58%, outperforming existing heuristic defenses.
Model Utility
A common concern with security hardening is "alignment tax"—the loss of general performance. Our results show that StruQ incurs virtually no utility loss. In AlpacaEval 1.0 benchmarks, the Llama-7B StruQ model achieved a win rate slightly higher (0.4%) than its undefended counterpart, proving that security can be achieved without sacrificing intelligence.
Conclusion
StruQ provides a robust, scalable solution to prompt injection by treating LLM interactions as structured queries rather than flat text. By combining a secure delimiter system with targeted negative training, StruQ effectively isolates the control plane from the data plane. Future work will focus on extending this framework to multi-turn conversational agents and exploring architectural modifications, such as attention masking, to further harden the model against white-box gradient-based attacks.
Summary updated