StruQ: Defending Against Prompt Injection with Structured Queries

Sizhe Chen, Julien Piet, Chawin Sitawarin, David A. Wagner

Introduction

Large Language Models (LLMs) have become central to modern software applications, yet they face a critical security vulnerability: prompt injection. Currently ranked as the top security risk for LLM applications, prompt injection occurs when a malicious user embeds instructions within untrusted input data to override the developer's original system prompt. This vulnerability stems from the "flat" nature of traditional LLM processing, where control instructions and user data are concatenated into a single string. Consequently, the model fails to distinguish between the control plane (developer commands) and the data plane (user input).

Comparison of prompt injection handling in standard vs. StruQ-defended LLMs.

The StruQ Framework

To address this fundamental design flaw, we introduce StruQ, a defense mechanism inspired by parameterized queries in SQL. StruQ enforces a strict separation between instructions and data through a two-component system: a secure front-end and a structured instruction-tuned model.

1. Secure Front-End and Recursive Filtering

The first line of defense is a secure front-end that transforms raw inputs into a structured format. Instead of using standard text delimiters (e.g., "### Instruction:"), which are easily spoofed, StruQ wraps inputs in special reserved tokens (e.g., [INST], [INPT], [RESP]). These tokens are added to the model's vocabulary and are inaccessible to standard user input.

To prevent "escape" attacks, the front-end employs a recursive data filter. This algorithm strips any attempts by a user to manually type reserved tokens or use control characters (like backspaces or carriage returns) to manipulate the model's internal state.

The StruQ architecture: from raw prompt and data to a structured query processed by a tuned LLM.

2. Structured Instruction Tuning

The core innovation of StruQ is Structured Instruction Tuning. Simply changing the input format is insufficient if the model still prioritizes any instruction it sees. We fine-tuned base models (such as Llama-7B and Mistral-7B) using a specialized dataset composition:

  • Clean Samples (50%): Standard instruction-following data to maintain general utility.
  • Attacked Samples (50%): Data containing "fake" instructions injected into the [INPT] block. During training, the model is penalized for following these injected commands and rewarded for adhering strictly to the developer's original prompt in the [INST] block.

This "negative training" teaches the LLM a crucial semantic rule: text contained within the data channel must be treated as passive information, never as executable commands.

Evaluation and Results

We evaluated StruQ across three dimensions: manual injection attacks, complex automated attacks, and model utility.

Security Performance

StruQ demonstrates remarkable resilience against a wide array of attacks. For manual prompt injections—the most common threat—the success rate dropped from near-certainty in undefended models to 0% in StruQ-defended versions.

Against more sophisticated threats, such as "Completion" attacks (where attackers simulate a model response to terminate the current task), StruQ maintained a success rate below 2%. Even against state-of-the-art optimization-based attacks like TAP (Tree-of-Attacks with Pruning), Llama's vulnerability plummeted from 97% to 9%. While white-box gradient attacks (GCG) remain a challenge, StruQ still reduced the success rate from 97% to 58%, outperforming existing heuristic defenses.

Model Utility

A common concern with security hardening is "alignment tax"—the loss of general performance. Our results show that StruQ incurs virtually no utility loss. In AlpacaEval 1.0 benchmarks, the Llama-7B StruQ model achieved a win rate slightly higher (0.4%) than its undefended counterpart, proving that security can be achieved without sacrificing intelligence.

Conclusion

StruQ provides a robust, scalable solution to prompt injection by treating LLM interactions as structured queries rather than flat text. By combining a secure delimiter system with targeted negative training, StruQ effectively isolates the control plane from the data plane. Future work will focus on extending this framework to multi-turn conversational agents and exploring architectural modifications, such as attention masking, to further harden the model against white-box gradient-based attacks.

Summary updated

Abstract

Recent advances in Large Language Models (LLMs) enable exciting LLM-integrated applications, which perform text-based tasks by utilizing their advanced language understanding capabilities. However, as LLMs have improved, so have the attacks against them. Prompt injection attacks are an important threat: they trick the model to deviate from the original application's instructions and instead follow user directives. These attacks rely on the LLM's ability to follow instructions and inability to separate the prompts and user data. We introduce structured queries, a general approach to tackle this problem. Structured queries separate prompts and data into two channels. We implement a system that supports structured queries. This system is made of (1) a secure front-end that formats a prompt and user data into a special format, and (2) a specially trained LLM that can produce high-quality outputs from these inputs. The LLM is trained using a novel fine-tuning strategy: we convert a base (non-instruction-tuned) LLM to a structured instruction-tuned model that will only follow instructions in the prompt portion of a query. To do so, we augment standard instruction tuning datasets with examples that also include instructions in the data portion of the query, and fine-tune the model to ignore these. Our system significantly improves resistance to prompt injection attacks, with little or no impact on utility. Our code is released at https://github.com/Sizhe-Chen/PromptInjectionDefense.
BibTeX
@inproceedings{chen2025struq,
  author = {Sizhe Chen and Julien Piet and Chawin Sitawarin and David A. Wagner},
  title = {{StruQ: Defending Against Prompt Injection with Structured Queries}},
  booktitle = {USENIX Security},
  year = {2025},
}
Figure