Large language models trained on a mixture of NLP tasks that are converted into a text-to-text format using prompts can generalize into novel forms of language and handle novel tasks. While prompt engineering often focuses on achieving superior performance, this research investigates whether the way an input is encoded affects the social biases promoted in outputs. Specifically, we study T0, a large-scale multi-task model, and compare two semantically equivalent input forms: Question Answering (QA) and Natural Language Inference (NLI).
The Impact of Task Format on Bias
In prompt-based learning, researchers typically optimize for accuracy. However, LLMs inherit social biases from their training data. This paper asks a critical question: Does the task format itself influence the level of bias? If the same information is presented as a "Who did X?" question (QA) versus a "Does sentence A imply sentence B?" statement (NLI), does the model's internal prejudice fluctuate?
Methodology and Datasets
The study utilized T0pp (an 11-billion parameter model) to compare semantically equivalent inputs across two formats. To ensure a robust comparison, the researchers used two primary benchmarks:
- BBQ (Bias Benchmark for QA): An existing benchmark covering 11 categories (gender, race, religion, etc.). The authors converted this into an NLI format (BBQ → NLI).
- BBNLI (Bias Benchmark for NLI): A newly introduced dataset featuring 3,642 unique pairs across 16 subtopics, capturing harmful stereotypes and cognitive biases. This was also converted into a QA format (BBNLI → QA).
Key Insights: Familiarity Breeds Bias
The quantitative gap between formats is striking. Bias scores range from -100 to 100 (where 0 is unbiased):
- BBNLI Dataset: The average bias score in NLI format was 6.79, but jumped to 19.41 in QA format.
- BBQ Dataset (Gender): The QA format showed a bias score of 43.59, while the NLI format dropped to 4.49.
- BBQ Dataset (Religion): QA bias was 37.16, while NLI bias was 13.98.
The results suggest that the model is consistently more biased when using the QA format—a format it saw extensively during its multi-task training. When forced into the NLI format (a "novel" task for this model), it acted more neutrally. Simply changing the task structure had a massive impact, far outweighing minor changes in prompt wording.
Conclusion and Implications
This research proves that task format matters significantly in the manifestation of social bias. In QA, the model often feels "forced" to pick a name even when the correct answer is "Unknown," whereas in NLI, it is more likely to correctly identify that a premise does not entail a biased hypothesis.
Future work in building multi-task models should consider "holding out" popular task formats during training. If models are only tested on formats they were trained on, we may be overestimating their performance while underestimating their potential for social harm.
Summary updated