Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions

Pouya Pezeshkpour, Estevam Hruschka

Introduction

Multiple-choice questions (MCQs) are the standard for evaluating Large Language Models (LLMs), measuring everything from common sense to specialized legal reasoning. A fundamental assumption in these benchmarks is that if a model possesses the necessary knowledge, the specific ordering of answer options—whether the correct choice is labeled (A), (B), or (E)—should not influence its response.

However, research presented at NAACL 2024 reveals a significant flaw in this assumption. LLMs exhibit extreme sensitivity to the internal arrangement of answer options, leading to what the authors call a "sensitivity gap."

An example of an LLM providing a correct response in the original order but a wrong response after the options are reordered.

Methodology

The study systematically evaluated three prominent models: GPT-4, InstructGPT (text-davinci-003), and Llama-2-13b. To quantify the impact of option ordering, the researchers measured the sensitivity gap—the difference between the maximum and minimum accuracy achieved by a model on the same set of questions across different permutations of options.

The evaluation spanned five diverse benchmarks to ensure domain-agnostic findings:

  • CSQA: Commonsense reasoning.
  • MMLU Subsets: Abstract Algebra, High School Chemistry, and Professional Law.
  • Big-Bench: Logical Deduction.

Beyond measurement, the authors explored calibration techniques, including Majority Voting (aggregating results from 10 random reorderings) and Multiple Evidence Calibration (MEC) (prompting for reasoning before selection).

Key Insights: Uncertainty and Positional Bias

The researchers identified that sensitivity is not random but is closely tied to model uncertainty.

  • The "Top-2" Tussle: Sensitivity peaks when a model is "torn" between its top two or three candidate answers. High confidence in a single answer typically results in order-invariance.
  • Positional Bias: When uncertain, models often default to a positional preference. For instance, GPT-4 showed a slight bias toward option "C" in technical domains.
  • Strategic Placement: Bias can be amplified by placing the top two candidates at the extremes (e.g., options A and E) and mitigated by placing them adjacently.
  • Few-shot Limitations: While few-shot demonstrations improve overall accuracy, they do not significantly reduce the sensitivity gap; models remain fickle regardless of provided examples.

Key Results

The findings underscore a massive performance variance across all tested models. The sensitivity gap reached as high as 85% for Llama-2-13b on certain tasks. Even GPT-4, while more robust, showed double-digit swings in accuracy based solely on option placement.

Table showing the vanilla performance alongside the minimum and maximum accuracy deviations achieved through option reordering.

Calibration proved effective: majority voting over 10 reorderings improved performance by up to 8 percentage points. However, MEC (Chain-of-Thought style reasoning) was a "double-edged sword," helping GPT-4 in reasoning tasks but degrading performance for InstructGPT, suggesting that reasoning-based calibration is not universally effective.

Conclusion

This research highlights that current LLM benchmarks may provide a "hallucinated" view of capability if they do not account for option-order sensitivity. While calibration techniques like majority voting offer a path forward, the root causes of positional bias—whether inherent in transformer architectures or a reflection of training data distributions—remain a critical area for future investigation.

Summary updated

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in various NLP tasks. However, previous works have shown these models are sensitive towards prompt wording, and few-shot demonstrations and their order, posing challenges to fair assessment of these models. As these models become more powerful, it becomes imperative to understand and address these limitations. In this paper, we focus on LLMs robustness on the task of multiple-choice questions -- commonly adopted task to study reasoning and fact-retrieving capability of LLMs. Investigating the sensitivity of LLMs towards the order of options in multiple-choice questions, we demonstrate a considerable performance gap of approximately 13% to 75% in LLMs on different benchmarks, when answer options are reordered, even when using demonstrations in a few-shot setting. Through a detailed analysis, we conjecture that this sensitivity arises when LLMs are uncertain about the prediction between the top-2/3 choices, and specific options placements may favor certain prediction between those top choices depending on the question caused by positional bias. We also identify patterns in top-2 choices that amplify or mitigate the model's bias toward option placement. We found that for amplifying bias, the optimal strategy involves positioning the top two choices as the first and last options. Conversely, to mitigate bias, we recommend placing these choices among the adjacent options. To validate our conjecture, we conduct various experiments and adopt two approaches to calibrate LLMs' predictions, leading to up to 8 percentage points improvement across different models and benchmarks.
BibTeX
@inproceedings{pezeshkpour2024large,
  author = {Pouya Pezeshkpour and Estevam Hruschka},
  title = {{Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-naacl.130},
}
Figure