Introduction
Text-to-SQL systems are becoming essential tools for bridging the gap between natural language and structured databases, finding applications in critical sectors like finance and healthcare. However, a significant concern remains: do these models inherit the social biases present in their underlying Large Language Models (LLMs)? This research introduces BiaSpider, a benchmark designed to expose how Text-to-SQL models make harmful stereotypical correlations and discriminative comparisons across demographics such as race, gender, and religion.
The Problem: Hidden Biases in "Clean" Data
A common assumption in the Text-to-SQL community is that training on neutral, "clean" datasets like Spider or WikiSQL prevents the model from exhibiting social bias. This paper challenges that notion, arguing that models integrally inherit biases from their pre-training phase. When these models are used to make decisions about individuals—such as determining credit eligibility—hidden biases can lead to discriminatory real-world outcomes.
Methodology: The BiaSpider Pipeline
To systematically uncover these biases, the researchers developed a novel paradigm to "trigger" biased responses. The process involves identifying human-centric tables, injecting judgmental modifiers (positive, negative, and comparative), and using GPT-3 to paraphrase neutral queries into "judgmental" ones.
The core of the evaluation is the Bias Score, which quantifies how often a model includes an unrequested demographic filter (e.g., WHERE ethnicity = 'Black') in its SQL output when prompted with a judgmental adjective like "terrible."
Categorizing Bias in Structured Data
The study identifies two primary manifestations of bias in Text-to-SQL:
- Stereotypical Correlation: The model incorrectly links a judgmental adjective to a specific demographic group.
- Discriminative Comparison: The model treats one demographic as inherently "better" or "worse" than another based on external metrics.
Key Results and Architecture Insights
Evaluating state-of-the-art models like RATSQL (BERT), UNISAR (BART), and PICARD (T5) revealed significant disparities:
- Architecture Matters: Encoder-only models (BERT-based) showed the highest bias, with scores often exceeding 40-50%. Encoder-Decoder models (T5, BART) were more robust, though still biased (~10-12%).
- The Accuracy Paradox: Models with similar task accuracy on the standard Spider dataset exhibited vastly different levels of social bias. This highlights that accuracy is an insufficient metric for evaluating model trustworthiness.
Conclusion
While training on neutral data is a standard practice, it is not a cure for inherited social bias. This research provides the tools to diagnose and categorize these biases, serving as a critical first step toward developing more equitable and trustworthy Text-to-SQL systems. Future work must now focus on developing effective de-biasing techniques for structured data tasks.
Summary updated