In the rapidly evolving landscape of Large Language Models (LLMs), researchers often focus on scaling parameters or refining training data. However, recent research reveals that performance on Reading Comprehension (RC) tasks is remarkably sensitive to a much simpler factor: prompt structure. By optimizing the order of inputs and applying basic emphasis markers, models can achieve accuracy gains of up to 31%, frequently allowing smaller models to outperform their much larger counterparts.
The Problem: Lack of Standardized Prompting
In a typical Reading Comprehension task, a model is provided with a context (such as a Wikipedia passage) and a question about that context. Despite the ubiquity of this task, there is no industry standard for how to present these inputs. Some researchers place the question first, while others lead with the context. This paper investigates whether this ordering matters and if "nudging" the model to pay attention to specific prompt segments can improve performance.
Methodology: Ordering and Emphasis
The study evaluated nine different LLMs, ranging from GPT-2 to Llama-2 13B, across three diverse benchmarks: Natural Questions (NQ), SQuAD v1.1, and AdversarialQA. The researchers tested two primary strategies:
- Input Ordering: Comparing "Question First" (Question: [Q] Context: [C]) against "Context First" (Context: [C] Question: [Q]).
- Emphasis Strategies:
- Marked Prompting (MP): A simple approach wrapping the question or context in special characters or tags, such as
*question*,"question", or<emphasize>question</emphasize>.
- Marked Prompting (MP): A simple approach wrapping the question or context in special characters or tags, such as
- Attention Steering (AS): A sophisticated method that manually down-weights the attention scores of tokens outside the target string during the model's inference process.
Key Results and Performance Gains
The impact of these simple structural changes was profound. Across almost all models and datasets, the "Context-First" rule held true: placing the context before the question significantly improved accuracy. On average, this swap resulted in a 13.46% accuracy increase. In the most extreme case—GPT-J on the Natural Questions dataset—accuracy jumped from 33.3% to 64.5% simply by changing the input order.
Emphasis also proved to be a powerful lever. Using the best emphasis method (typically the <emphasize> tag) provided accuracy improvements of up to 36% in specific settings.
Why It Works: Perplexity and Parametric Knowledge
The researchers used Perplexity to measure how "confusing" different prompt structures were to the models. They found that Context-First prompts showed an average perplexity reduction of 0.73, suggesting that LLMs find it easier to process a question once they have already ingested the relevant background information.
Furthermore, the study highlighted a critical distinction between Known and Unknown knowledge. Emphasis is most effective when the model does not already know the answer from its internal training data (parametric knowledge). When a fact is new to the model, these structural cues act as a vital guide for the model's attention.
Insights and Future Directions
One of the most surprising findings was that simple Marked Prompting was often just as effective as the more complex Attention Steering. This suggests that models are highly sensitive to surface-level formatting cues.
While this study focused on Reading Comprehension, it raises questions about whether similar rules apply to tasks like summarization or reasoning. The authors conclude by calling for a standardization of RC input presentation to ensure fair and consistent benchmarking across the NLP community. For practitioners, the takeaway is clear: before reaching for a larger model, try putting your context first and wrapping your key instructions in <emphasize> tags.
Summary updated