Introduction
Multiple-choice questions (MCQs) are the standard for evaluating Large Language Models (LLMs), measuring everything from common sense to specialized legal reasoning. A fundamental assumption in these benchmarks is that if a model possesses the necessary knowledge, the specific ordering of answer options—whether the correct choice is labeled (A), (B), or (E)—should not influence its response.
However, research presented at NAACL 2024 reveals a significant flaw in this assumption. LLMs exhibit extreme sensitivity to the internal arrangement of answer options, leading to what the authors call a "sensitivity gap."
Methodology
The study systematically evaluated three prominent models: GPT-4, InstructGPT (text-davinci-003), and Llama-2-13b. To quantify the impact of option ordering, the researchers measured the sensitivity gap—the difference between the maximum and minimum accuracy achieved by a model on the same set of questions across different permutations of options.
The evaluation spanned five diverse benchmarks to ensure domain-agnostic findings:
- CSQA: Commonsense reasoning.
- MMLU Subsets: Abstract Algebra, High School Chemistry, and Professional Law.
- Big-Bench: Logical Deduction.
Beyond measurement, the authors explored calibration techniques, including Majority Voting (aggregating results from 10 random reorderings) and Multiple Evidence Calibration (MEC) (prompting for reasoning before selection).
Key Insights: Uncertainty and Positional Bias
The researchers identified that sensitivity is not random but is closely tied to model uncertainty.
- The "Top-2" Tussle: Sensitivity peaks when a model is "torn" between its top two or three candidate answers. High confidence in a single answer typically results in order-invariance.
- Positional Bias: When uncertain, models often default to a positional preference. For instance, GPT-4 showed a slight bias toward option "C" in technical domains.
- Strategic Placement: Bias can be amplified by placing the top two candidates at the extremes (e.g., options A and E) and mitigated by placing them adjacently.
- Few-shot Limitations: While few-shot demonstrations improve overall accuracy, they do not significantly reduce the sensitivity gap; models remain fickle regardless of provided examples.
Key Results
The findings underscore a massive performance variance across all tested models. The sensitivity gap reached as high as 85% for Llama-2-13b on certain tasks. Even GPT-4, while more robust, showed double-digit swings in accuracy based solely on option placement.
Calibration proved effective: majority voting over 10 reorderings improved performance by up to 8 percentage points. However, MEC (Chain-of-Thought style reasoning) was a "double-edged sword," helping GPT-4 in reasoning tasks but degrading performance for InstructGPT, suggesting that reasoning-based calibration is not universally effective.
Conclusion
This research highlights that current LLM benchmarks may provide a "hallucinated" view of capability if they do not account for option-order sensitivity. While calibration techniques like majority voting offer a path forward, the root causes of positional bias—whether inherent in transformer architectures or a reflection of training data distributions—remain a critical area for future investigation.
Summary updated