TL;DR
Large Language Models (LLMs) often struggle with negation, leading to misinterpretations. This paper introduces novel prompt engineering techniques—Warning-based and Persona-based—that significantly improve LLMs' negation reasoning accuracy. They also propose a Negative Token Attention Score (NTAS) to quantify attention to negation words, finding it correlates better with performance than model size.
What Problem Does This Paper Address?
Negation is a fundamental aspect of human language, yet it poses a significant challenge for Large Language Models (LLMs). LLMs can misinterpret sentences containing negation, leading to incorrect truth condition assessments. While various methods have been explored to improve LLM robustness to negation, such as specialized training strategies or paraphrasing, there's a need for more accessible and effective techniques. This paper aims to address this gap by investigating how prompt engineering can enhance LLMs' ability to reason about negated statements, focusing on improving accuracy and understanding the underlying mechanisms.
Methods
The core of this research lies in prompt engineering. The authors propose two new genres of prompts:
- Warning-based Prompts (WP): These prompts aim to guide the LLM by explicitly warning it about the presence of negation and potential distractors.
- WP1 (Negation-Distraction): "Pay attention to any negation and distractors (sentences that don't make sense)."
- WP2 (Negation Spoonfeeding): "Pay attention to any negation (words like ‘not', 'no', 'never', etc.)."
- Persona-based Prompts (PP): These prompts assign a role or persona to the LLM to influence its reasoning.
- PP1 (Linguistic Expert): "You are a linguistic expert who understands the effect of negation words."
- PP2 (Negation Expert): "You are an expert in analysing negations in sentences."
These proposed prompts are appended to a baseline prompt ("Is the following statement True or False?") to create a more comprehensive input.
Additionally, the paper introduces the Negative Token Attention Score (NTAS). Inspired by previous work on negative attention, NTAS quantifies how much attention a model pays to specific negation tokens (like "no," "never," "not") within a sentence during inference. This metric helps to understand why certain prompts might be more effective.
The study also investigates prompt reordering to assess LLM robustness and the impact of different positional encoding schemes (Relative, Absolute, Rotary) on performance.
Datasets
The primary dataset used is This-is-not-a-Dataset introduced by García-Ferrero et al. (2023). This dataset comprises approximately 400,000 semi-automatically generated English sentences, with about two-thirds containing various forms of negation. It is structured into 11 distinct patterns designed to test different aspects of negation understanding, drawing from WordNet relations and semantic roles. The authors specifically use the test set, which contains around 90,000 sentences.
Key Insights and Ideas
- Prompt Engineering is Powerful: Even simple prompt variations can significantly boost LLM performance on negation tasks, suggesting that LLMs are highly sensitive to how instructions are framed.
- NTAS as a Diagnostic Tool: The Negative Token Attention Score (NTAS) provides a quantifiable measure of how well LLMs are attending to negation. This is a valuable tool for understanding model behavior beyond simple accuracy metrics.
- Attention to Negation Matters More Than Size (Within a Family): For models within the same family (e.g., different sizes of IBM Granite), higher NTAS scores correlated more strongly with better accuracy than model size alone. This implies that focusing attention on the right linguistic elements can be more critical than sheer scale.
- Positional Encoding Impacts Robustness: The way LLMs handle positional information (Absolute, Relative, Rotary) affects their consistency when prompt order is changed. Models with Relative positional encoding appear more robust to prompt reordering.
- Distractors are Tricky: LLMs struggle with sentences containing distractors that subtly alter meaning, but specific prompts (like Negation-Distraction) can significantly improve performance on these challenging cases.
Key Results
- The proposed Warning-based and Persona-based prompts improved overall absolute accuracy by up to 3.17% and distractor negation accuracy by up to 25.14% over baseline prompts.
- The Negation-Distraction prompt showed a notable improvement in distractor negation accuracy for IBM-Granite-3.0-2B (25.14%) and Qwen2.5-14B (10.9%).
- Models with Relative positional encoding (e.g., FLAN-T5) demonstrated stable performance even when prompts were reordered, unlike models with Absolute or Rotary encodings.
- The study found that larger models do not always outperform smaller ones in negation tasks. Instead, a higher NTAS score was a better indicator of performance within a model family. For instance, IBM-Granite 2B outperformed the 3B model with a higher NTAS and accuracy.
Limitations and Future Work
The authors acknowledge several limitations:
- Focus on First-Token Probabilities: The study primarily analyzed first-token probabilities, which reduces inference time but might not capture reasoning errors that occur later in a model's response. Future work could involve a more detailed evaluation of multi-token completions.
- Limited Prompting Strategies: While significant performance variations were observed, the study explored only a limited set of prompting strategies. Future research could investigate a wider range of prompt formulations, potentially using reinforcement learning or adversarial testing.
- Model Family Constraints: The analysis was limited to specific architectures and families of models. Different pretraining objectives or fine-tuning techniques could influence performance in ways not captured by this study. Extending the analysis to a broader set of models is suggested.
Future work directions include:
- Pattern-wise Analysis: Conducting a thorough analysis of the eleven linguistic patterns in the dataset to identify specific areas where LLMs excel or struggle.
- Alternative Labels: Exploring the use of Yes/No labels instead of True/False for negation reasoning.
- Multilingual Negation: Investigating multilingual negation reasoning and the transferability of effective prompts across languages.
Summary updated