Abstract
Large Language Models (LLMs) have demonstrated remarkable capabilities in various NLP tasks. However, previous works have shown these models are sensitive towards prompt wording, and few-shot demonstrations and their order, posing challenges to fair assessment of these models. As these models become more powerful, it becomes imperative to understand and address these limitations. In this paper, we focus on LLMs robustness on the task of multiple-choice questions -- commonly adopted task to study reasoning and fact-retrieving capability of LLMs. Investigating the sensitivity of LLMs towards the order of options in multiple-choice questions, we demonstrate a considerable performance gap of approximately 13% to 75% in LLMs on different benchmarks, when answer options are reordered, even when using demonstrations in a few-shot setting. Through a detailed analysis, we conjecture that this sensitivity arises when LLMs are uncertain about the prediction between the top-2/3 choices, and specific options placements may favor certain prediction between those top choices depending on the question caused by positional bias. We also identify patterns in top-2 choices that amplify or mitigate the model's bias toward option placement. We found that for amplifying bias, the optimal strategy involves positioning the top two choices as the first and last options. Conversely, to mitigate bias, we recommend placing these choices among the adjacent options. To validate our conjecture, we conduct various experiments and adopt two approaches to calibrate LLMs' predictions, leading to up to 8 percentage points improvement across different models and benchmarks.
Introduction
Multiple-choice questions (MCQs) are the industry standard for evaluating the reasoning and factual knowledge of Large Language Models (LLMs). However, a significant challenge persists: if a model’s answer changes simply because "Option A" was moved to "Option C," benchmarks may be measuring prompt-formatting luck rather than actual intelligence. This study investigates the "sensitivity gap"—the difference between a model's best and worst possible performance across different option orderings—to understand why this happens and how to improve model robustness.
Methods and Datasets
The researchers employed a multi-stage analytical approach across five diverse MCQ benchmarks: CSQA (commonsense reasoning), MMLU (Abstract Algebra, High School Chemistry, Professional Law), and Big-Bench (Logical Deduction).
- Sensitivity Quantization: Measuring GPT-4 and InstructGPT performance across benchmarks to calculate the gap between best-case (oracle) and worst-case orderings.
- Uncertainty Probing: Using Hits@N metrics and self-verification to determine if the model "knows" it is guessing.
- Pattern Analysis: Testing option placement patterns to identify which arrangements amplify or mitigate bias.
- Calibration Testing: Evaluating strategies like Majority Voting (picking the most frequent answer across 10 random shuffles) and Multiple Evidence Calibration (MEC) (Chain-of-Thought style explanations).
Analysis of Positional Bias
The study reveals that positional bias is not random; it is triggered by model uncertainty. When a model is torn between the top 2 or 3 most likely choices, the position of those options dictates the final selection.
Key findings regarding placement include:
- The "Bookend" Effect: Positioning the top two candidate answers as the first and last options (e.g., A and E) maximizes bias and is most likely to "trick" the model.
- Safety in Adjacency: Placing the most likely choices next to each other helps the model compare them more effectively, thereby mitigating bias.
- Scale Limitations: Even GPT-4, while more robust than InstructGPT, exhibits a 13.1% sensitivity gap on tasks where its base accuracy exceeds 90%, proving that high performance does not guarantee robustness.
Key Results
The experimental results highlight massive performance swings. In zero-shot settings, reordering options caused performance gaps ranging from 13% to 75%. Notably, adding few-shot examples (up to 5-shot) improved overall accuracy but did little to close the sensitivity gap.
Calibration efforts yielded mixed results. Implementing a majority vote over 10 random reorders improved performance by up to 8 percentage points. However, the MEC approach (asking for explanations) actually decreased performance for InstructGPT, suggesting that "hallucinating" an explanation for an uncertain choice can reinforce incorrect answers.
Conclusion
This research demonstrates that LLMs are highly sensitive to option ordering in MCQs, primarily due to positional bias during moments of uncertainty. While majority voting provides a viable path to calibration, it is computationally expensive. Future work is required to develop more efficient calibration methods and investigate the root of these biases within the massive training corpora used for these models.
Summary updated