Uncovering and Categorizing Social Biases in Text-to-SQL

Yan Liu, Zhe Su, Xiaokang Chen, Elliott Ash, Jian-Guang Lou

Introduction

Text-to-SQL systems are becoming essential tools for bridging the gap between natural language and structured databases, finding applications in critical sectors like finance and healthcare. However, a significant concern remains: do these models inherit the social biases present in their underlying Large Language Models (LLMs)? This research introduces BiaSpider, a benchmark designed to expose how Text-to-SQL models make harmful stereotypical correlations and discriminative comparisons across demographics such as race, gender, and religion.

The Problem: Hidden Biases in "Clean" Data

A common assumption in the Text-to-SQL community is that training on neutral, "clean" datasets like Spider or WikiSQL prevents the model from exhibiting social bias. This paper challenges that notion, arguing that models integrally inherit biases from their pre-training phase. When these models are used to make decisions about individuals—such as determining credit eligibility—hidden biases can lead to discriminatory real-world outcomes.

Methodology: The BiaSpider Pipeline

To systematically uncover these biases, the researchers developed a novel paradigm to "trigger" biased responses. The process involves identifying human-centric tables, injecting judgmental modifiers (positive, negative, and comparative), and using GPT-3 to paraphrase neutral queries into "judgmental" ones.

The BiaSpider pipeline for generating biased benchmarks and evaluating Text-to-SQL models.

The core of the evaluation is the Bias Score, which quantifies how often a model includes an unrequested demographic filter (e.g., WHERE ethnicity = 'Black') in its SQL output when prompted with a judgmental adjective like "terrible."

Categorizing Bias in Structured Data

The study identifies two primary manifestations of bias in Text-to-SQL:

  1. Stereotypical Correlation: The model incorrectly links a judgmental adjective to a specific demographic group.
  2. Discriminative Comparison: The model treats one demographic as inherently "better" or "worse" than another based on external metrics.
Examples of Stereotypical Correlation and Discriminative Comparison in Text-to-SQL outputs.

Key Results and Architecture Insights

Evaluating state-of-the-art models like RATSQL (BERT), UNISAR (BART), and PICARD (T5) revealed significant disparities:

  • Architecture Matters: Encoder-only models (BERT-based) showed the highest bias, with scores often exceeding 40-50%. Encoder-Decoder models (T5, BART) were more robust, though still biased (~10-12%).
  • The Accuracy Paradox: Models with similar task accuracy on the standard Spider dataset exhibited vastly different levels of social bias. This highlights that accuracy is an insufficient metric for evaluating model trustworthiness.

Conclusion

While training on neutral data is a standard practice, it is not a cure for inherited social bias. This research provides the tools to diagnose and categorize these biases, serving as a critical first step toward developing more equitable and trustworthy Text-to-SQL systems. Future work must now focus on developing effective de-biasing techniques for structured data tasks.

Summary updated

Abstract

Large pre-trained language models are acknowledged to carry social bias towards different demographics, which can further amplify existing stereotypes in our society and cause even more harm. Text-to-SQL is an important task, models of which are mainly adopted by administrative industries, where unfair decisions may lead to catastrophic consequences. However, existing Text-to-SQL models are trained on clean, neutral datasets, such as Spider and WikiSQL. This, to some extent, cover up social bias in models under ideal conditions, which nevertheless may emerge in real application scenarios. In this work, we aim to uncover and mitigate social bias in Text-to-SQL models. We summarize the categories of social bias that may occur in structural data for Text-to-SQL models. We build test benchmarks and reveal that models with similar task accuracy can contain social bias at very different rates. We show how to take advantage of our methodology to assess and mitigate social bias in the downstream Text-to-SQL task.
BibTeX
@inproceedings{liu2023uncovering,
  author = {Yan Liu and Yan Gao and Zhe Su and Xiaokang Chen and Elliott Ash and Jian-Guang Lou},
  title = {{Uncovering and Categorizing Social Biases in Text-to-SQL}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.759},
}
Figure