Introduction
While most AI benchmarks are designed to be perfectly clear, real-world human requests are often riddled with ambiguity. This paper introduces AmbiBench, a new benchmark to test how well humans and Large Language Models (LLMs) resolve "task ambiguity"—situations where multiple rules could explain the same set of examples. The researchers found that while standard LLMs struggle, those trained with human feedback can match or even outperform humans at deducing the intended task.
What Problem Does This Paper Address?
In a typical NLP benchmark, the goal is crystal clear: "Summarize this text" or "Translate this to French." However, in the real world, users often provide underspecified instructions. If you give a model three examples of sentences labeled "X" that all happen to contain both an animal and a mention of the outdoors, and then ask it to label a fourth sentence, which rule should it follow? Does "X" mean "contains an animal" or "contains an outdoor location"?
This is task ambiguity. Most current AI evaluation focuses on how well a model performs a known task, but this paper shifts the focus to how well a model infers which task it should be doing in the first place. This is critical for safety and alignment; if a model latches onto the wrong pattern (a "shortcut"), it might fail catastrophically when deployed in a new environment.
Datasets: Introducing AmbiBench
The authors introduced AmbiBench, a benchmark consisting of six binary classification tasks. Each task involves detecting a simple linguistic feature (e.g., is there a proper noun? Is there negation?).
The "trick" is that every sentence in AmbiBench is programmatically generated to contain two distinct features. For example:
- Sentence: "The researcher is in the museum."
- Feature A: Human subject ("researcher").
- Feature B: Indoor location ("museum").
To succeed, the agent must use instructions or a series of examples to determine which of those two features is the "salient" one for the current task.
Methods
The authors evaluated a diverse range of models—including OpenAI’s GPT-3 series (from the tiny ada to the 175B davinci), AI21’s Jurassic-1, and instruction-tuned models like Flan-T5—against human participants recruited via Prolific.
They tested two primary ways of resolving ambiguity:
- Natural Language Instructions: Providing "Informative" instructions (explicitly stating the rule) vs. "Uninformative" instructions (redacting the rule).
- In-Context Learning: Providing a sequence of 1 to 20 labeled examples to see if the model can "Bayesian-style" narrow down the correct rule as more data points are added.
Finally, they explored meta-learning: finetuning a standard model (davinci) on a small set of ambiguous examples from some tasks to see if it learns a general ability to resolve ambiguity in entirely new, unseen tasks.
Key Insights and Ideas
- The "Human Feedback" Edge: Scaling a model to 175 billion parameters isn't enough to solve ambiguity. However, models trained with Human Feedback Data (HFD)—like
text-davinci-002—showed a massive jump in performance. This suggests that learning from humans inherently involves learning how to navigate the ambiguous way humans communicate. - Models as Better Bayesians: Surprisingly, the best HFD models actually outperformed humans when given 20 examples. While humans might get distracted or fatigued, the models were more consistent at narrowing down the possible rules as more evidence appeared.
- The Verbalization Gap: Even when a model performed a task perfectly (showing it "knew" the rule), it often failed to explain that rule in words when asked. This suggests that "doing" and "understanding" are still somewhat decoupled in LLMs.
- Ambiguity is Teachable: By finetuning a standard model on just 272 ambiguous examples, the researchers were able to close the performance gap between a "base" model and a "human-feedback" model. This implies that "learning to disambiguate" is a high-level skill that generalizes across different types of tasks.
Key Results
- Instruction Following: On informative instructions, humans achieved near-perfect accuracy. The best HFD model (
text-davinci-003) was close behind, while non-HFD models (like the basedavinciorj1-jumbo) performed significantly worse. - Few-Shot Scaling: As the number of examples increased from 1 to 20, the accuracy of
text-davinci-002rose steadily, eventually surpassing human accuracy (which hovered around 75-80% due to fatigue or oversight).
- Generalization: Models finetuned on ambiguous prompts from four tasks showed immediate, dramatic improvements on two completely different, held-out tasks, proving that the model learned a strategy for resolving ambiguity rather than just memorizing the specific tasks.
Limitations and Future Work
The authors acknowledge several limitations in their study:
- Complexity: AmbiBench uses very simple, templated sentences. Real-world ambiguity is often much more "messy" and high-dimensional.
- Human Factors: Human participants may have underperformed due to the repetitive nature of the 20-example tests, leading to fatigue that models don't experience.
- Binary Choice: The study focused on choosing between two possible tasks. In reality, a single prompt could be interpreted in an infinite number of ways.
Future Directions: The researchers suggest exploring task ambiguity in multimodal settings (images/video) and investigating how to make models better at "asking for clarification" when they detect they are in an ambiguous situation.
Summary updated