Task Ambiguity in Humans and Language Models

Alex Tamkin, Kunal Handa, Avash Shrestha, Noah GoodmanICLR 2023 · OpenReview

Introduction

While most AI benchmarks are designed to be perfectly clear, real-world human requests are often riddled with ambiguity. This paper introduces AmbiBench, a new benchmark to test how well humans and Large Language Models (LLMs) resolve "task ambiguity"—situations where multiple rules could explain the same set of examples. The researchers found that while standard LLMs struggle, those trained with human feedback can match or even outperform humans at deducing the intended task.

What Problem Does This Paper Address?

In a typical NLP benchmark, the goal is crystal clear: "Summarize this text" or "Translate this to French." However, in the real world, users often provide underspecified instructions. If you give a model three examples of sentences labeled "X" that all happen to contain both an animal and a mention of the outdoors, and then ask it to label a fourth sentence, which rule should it follow? Does "X" mean "contains an animal" or "contains an outdoor location"?

This is task ambiguity. Most current AI evaluation focuses on how well a model performs a known task, but this paper shifts the focus to how well a model infers which task it should be doing in the first place. This is critical for safety and alignment; if a model latches onto the wrong pattern (a "shortcut"), it might fail catastrophically when deployed in a new environment.

Datasets: Introducing AmbiBench

The authors introduced AmbiBench, a benchmark consisting of six binary classification tasks. Each task involves detecting a simple linguistic feature (e.g., is there a proper noun? Is there negation?).

The "trick" is that every sentence in AmbiBench is programmatically generated to contain two distinct features. For example:

  • Sentence: "The researcher is in the museum."
  • Feature A: Human subject ("researcher").
  • Feature B: Indoor location ("museum").

To succeed, the agent must use instructions or a series of examples to determine which of those two features is the "salient" one for the current task.

An example of task ambiguity in AmbiBench, where an agent must deduce the intended category from examples.

Methods

The authors evaluated a diverse range of models—including OpenAI’s GPT-3 series (from the tiny ada to the 175B davinci), AI21’s Jurassic-1, and instruction-tuned models like Flan-T5—against human participants recruited via Prolific.

They tested two primary ways of resolving ambiguity:

  1. Natural Language Instructions: Providing "Informative" instructions (explicitly stating the rule) vs. "Uninformative" instructions (redacting the rule).
  2. In-Context Learning: Providing a sequence of 1 to 20 labeled examples to see if the model can "Bayesian-style" narrow down the correct rule as more data points are added.

Finally, they explored meta-learning: finetuning a standard model (davinci) on a small set of ambiguous examples from some tasks to see if it learns a general ability to resolve ambiguity in entirely new, unseen tasks.

Key Insights and Ideas

  • The "Human Feedback" Edge: Scaling a model to 175 billion parameters isn't enough to solve ambiguity. However, models trained with Human Feedback Data (HFD)—like text-davinci-002—showed a massive jump in performance. This suggests that learning from humans inherently involves learning how to navigate the ambiguous way humans communicate.
  • Models as Better Bayesians: Surprisingly, the best HFD models actually outperformed humans when given 20 examples. While humans might get distracted or fatigued, the models were more consistent at narrowing down the possible rules as more evidence appeared.
  • The Verbalization Gap: Even when a model performed a task perfectly (showing it "knew" the rule), it often failed to explain that rule in words when asked. This suggests that "doing" and "understanding" are still somewhat decoupled in LLMs.
  • Ambiguity is Teachable: By finetuning a standard model on just 272 ambiguous examples, the researchers were able to close the performance gap between a "base" model and a "human-feedback" model. This implies that "learning to disambiguate" is a high-level skill that generalizes across different types of tasks.

Key Results

  • Instruction Following: On informative instructions, humans achieved near-perfect accuracy. The best HFD model (text-davinci-003) was close behind, while non-HFD models (like the base davinci or j1-jumbo) performed significantly worse.
  • Few-Shot Scaling: As the number of examples increased from 1 to 20, the accuracy of text-davinci-002 rose steadily, eventually surpassing human accuracy (which hovered around 75-80% due to fatigue or oversight).
Accuracy of humans and various models as the number of in-context examples increases. Models trained with human feedback eventually surpass human performance.
  • Generalization: Models finetuned on ambiguous prompts from four tasks showed immediate, dramatic improvements on two completely different, held-out tasks, proving that the model learned a strategy for resolving ambiguity rather than just memorizing the specific tasks.

Limitations and Future Work

The authors acknowledge several limitations in their study:

  • Complexity: AmbiBench uses very simple, templated sentences. Real-world ambiguity is often much more "messy" and high-dimensional.
  • Human Factors: Human participants may have underperformed due to the repetitive nature of the 20-example tests, leading to fatigue that models don't experience.
  • Binary Choice: The study focused on choosing between two possible tasks. In reality, a single prompt could be interpreted in an infinite number of ways.

Future Directions: The researchers suggest exploring task ambiguity in multimodal settings (images/video) and investigating how to make models better at "asking for clarification" when they detect they are in an ambiguous situation.

Summary updated

Abstract

Language models have recently achieved strong performance across a wide range of NLP benchmarks. However, real world tasks are often poorly specified, and agents must deduce the intended behavior from a combination of context, instructions, and examples. We investigate how both humans and models behave in the face of such task ambiguity by proposing AmbiBench, a new benchmark of six ambiguously-specified classification tasks. We evaluate humans and models on AmbiBench by seeing how well they identify the intended task using 1) instructions with varying degrees of ambiguity, and 2) different numbers of labeled examples. We find that the combination of model scaling (to 175B parameters) and reinforcement learning from human feedback (RLHF) enables models to approach or exceed the accuracy of human participants across tasks, but that either one of these alone is not sufficient. In addition, we show how to dramatically improve the accuracy of language models trained without RLHF by finetuning on a small number of ambiguous in-context examples, providing a promising direction for teaching models to generalize well in the face of ambiguity.
BibTeX
@inproceedings{tamkin2023task,
  author = {Alex Tamkin and Kunal Handa and Avash Shrestha and Noah Goodman},
  title = {{Task Ambiguity in Humans and Language Models}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=QrnDe_9ZFd8},
}
Figure