Introduction
Large Language Models (LLMs) like GPT-3.5 and GPT-4 have demonstrated remarkable capabilities across diverse tasks. However, they remain notoriously "brittle" when faced with adversarial inputs. Small, semantic-preserving perturbations—such as typos, synonym swaps, or character-level changes—can cause a model's performance to plummet. For instance, in aspect-based sentiment tasks, simply flipping the sentiment of a non-target word can lead to an accuracy drop of nearly 35% in GPT-3.5.
Traditional defenses often rely on adversarial training, which requires updating model parameters. This is not feasible for "black-box" models accessed via APIs. To address this, the RoCoIns (Robust Code Instructions) framework introduces a novel prompting strategy that builds robustness directly into instruction design and in-context learning (ICL) without requiring fine-tuning.
The RoCoIns Framework
RoCoIns utilizes two primary techniques to stabilize model performance against adversarial noise:
1. Code-Style Instructions
Instead of providing instructions in natural language (e.g., "Is this sentence positive or negative?"), the task is encapsulated within a non-executable Python-style class. This structure leverages the inherent logic and lack of ambiguity in code. The prompt typically includes a class name as a task summary, docstrings for parameter definitions, and descriptive function names (like is_positive()) to guide the model's internal reasoning.
2. Adversarial Context Method
Drawing on the theory that in-context learning (ICL) acts as a form of "implicit fine-tuning," RoCoIns includes both clean and adversarial samples in few-shot demonstrations. By observing how a perturbed sentence should still map to its original correct label, the model learns to ignore adversarial noise during inference. This "adversarial demonstration" proved to be the most significant driver of robustness gains in the study.
Experimental Setup and Datasets
The researchers evaluated RoCoIns across eight robustness-focused datasets from two major benchmarks:
- AdvGLUE: An adversarial version of the GLUE benchmark, covering tasks such as sentiment analysis (SST-2), question parity (QQP), and natural language inference (MNLI, QNLI, RTE).
- Restaurant-T: An aspect-based sentiment analysis (ABSA) dataset featuring transformations like reversing target sentiment (
RevTgt), reversing non-target sentiment (RevNon), and adding distracting aspects (AddDiff).
Key Insights
Alignment with Pre-training
Perplexity analysis indicated that code-style instructions result in lower perplexity for models like CodeT5 compared to natural language prompts. This suggests that converting instructions into code better aligns the prompt with the high-quality code data LLMs encountered during pre-training, making the task more "natural" for the model to process.
Attention and Reasoning
Gradient visualization (Saliency Maps) revealed that natural language prompts often cause models to fixate on irrelevant "stop words" (like "the") when faced with adversarial inputs. In contrast, code-style prompts help the model maintain focus on core sentiment-bearing words. Interestingly, the study found that Chain-of-Thought (CoT) prompting—often a gold standard for reasoning—can actually decrease robustness, as complex reasoning steps can lead the model down a "hallucination path" when triggered by adversarial noise.
Results and Discussion
RoCoIns demonstrated significant improvements across the board for gpt-3.5-turbo and text-davinci-003:
- Accuracy Boost: Average accuracy on adversarial samples improved by 5.68% for
gpt-3.5-turbo. - Attack Mitigation: The average Attack Success Rate (ASR) dropped by 5.66 points.
- Scaling with Shots: While code-style instructions provide a baseline improvement in zero-shot scenarios, the combination with adversarial context in few-shot settings yielded the highest performance.
Conclusion
RoCoIns provides a practical, fine-tuning-free method to enhance LLM robustness. By aligning instructions with the structural logic of code and providing adversarial context, the framework helps models ignore irrelevant perturbations. Future work will focus on automating the generation of these robust code structures for a wider variety of NLP tasks and exploring their effectiveness on smaller, open-source models.
Summary updated