Confronting LLMs with Traditional ML: Rethinking the Fairness of Large Language Models in Tabular Classifications

Yanchen Liu, Srishti Gautam, Jiaqi Ma, Himabindu Lakkaraju

Abstract

Recent literature has suggested the potential of using large language models (LLMs) to make classifications for tabular tasks. However, LLMs have been shown to exhibit harmful social biases that reflect the stereotypes and inequalities present in society. This study investigates the sources of information LLMs draw upon for tabular classification, the extent of social bias influence, and the implications for fairness. Through experiments comparing GPT-3.5-turbo with traditional machine learning models like Random Forest (RF) and Neural Networks (NN), we demonstrate that LLMs inherit significant social biases from their pre-training data. While in-context learning and fine-tuning offer moderate mitigation, fairness gaps often remain larger than those in traditional models. Notably, label-flipping of in-context examples can significantly reduce these biases, highlighting their inherent nature within LLMs.

Introduction

Tabular data is foundational to critical decision-making in sectors such as finance, healthcare, and law. While specialized machine learning (ML) models have traditionally handled these tasks, there is a growing trend toward using Large Language Models (LLMs) by serializing tabular data into natural language prompts. However, LLMs are trained on vast internet corpora containing human stereotypes and societal inequalities. This research explores whether these "semantic priors" cause LLMs to produce biased predictions in high-stakes tabular classifications, such as credit risk or recidivism, and how they compare to traditional "tabula rasa" ML models.

Methodology

The study evaluates GPT-3.5-turbo against two traditional benchmarks: Random Forests (RF) and a 3-layer Neural Network (NN). Tabular data was converted into descriptive strings (serialization) to be processed by the LLM. The models were tested in three primary settings:

  1. Zero-Shot: Direct prediction without prior examples.
  2. Few-Shot (In-Context Learning): Providing 50 labeled examples within the prompt.
  3. Fine-tuning: Training on the full task-specific dataset.

Additionally, the researchers explored "Label Flipping" in the few-shot setting—intentionally providing examples with reversed labels—and "Resampled Fine-tuning" to assess standard bias mitigation efficacy.

Experimental Setup

Three benchmark datasets were used:

  • Adult Income: Predicting income >$50k/year (Protected attribute: Sex).
  • German Credit: Classifying credit risk (Protected attribute: Sex).
  • COMPAS: Predicting criminal recidivism (Protected attribute: Race).

Key Findings: The "Internet Baggage" Problem

A central discovery is that LLMs arrive with "pre-loaded" biases. Unlike traditional models that learn only from the provided dataset, LLMs utilize semantic priors. Even when protected attributes were removed, fairness gaps persisted, suggesting the model uses other features as proxies for stereotypes.

Fairness comparison between GPT-3.5 Zero-Shot and traditional ML models (RF, NN) across three datasets.

As shown in the figure above, traditional ML models (RF and NN) frequently achieved better fairness scores (lower absolute difference in metrics like Statistical Parity and Equality of Opportunity) than zero-shot LLMs. This is attributed to traditional models lacking preconceived notions about social groups outside the specific training data.

Bias Mitigation Strategies

While fine-tuning proved to be the most effective standard method for reducing bias, bringing GPT-3.5's metrics closer to traditional models, other techniques showed mixed results. Standard resampling (oversampling minority groups), a staple in traditional ML, was inconsistent for LLMs.

Impact of Few-Shot learning and Label Flipping (LF) on fairness metrics.

The most striking mitigation result came from Label Flipping. By providing few-shot examples where disadvantaged groups had positive outcomes, the fairness gap was significantly reduced. For instance, on the Adult dataset, the Equality of Opportunity (EoO) gap dropped from 0.483 to 0.037. This diagnostic confirms that the bias is inherent to the model's weights rather than just reflecting the task-specific data. However, this improvement often comes at the cost of overall prediction accuracy.

Conclusion

LLMs present a sobering reality for tabular data tasks: they often inherit and amplify societal biases, performing worse on fairness metrics than traditional models. While techniques like fine-tuning and label-flipping can mitigate these effects, the inherent biases within LLMs remain a significant hurdle for high-stakes applications. Future work should explore a wider variety of LLMs and advanced prompting strategies like Chain-of-Thought to further address these systemic issues.

Summary updated

Abstract

Recent literature has suggested the potential of using large language models (LLMs) to make classifications for tabular tasks. However, LLMs have been shown to exhibit harmful social biases that reflect the stereotypes and inequalities present in society. To this end, as well as the widespread use of tabular data in many high-stake applications, it is important to explore the following questions: what sources of information do LLMs draw upon when making classifications for tabular tasks; whether and to what extent are LLM classifications for tabular data influenced by social biases and stereotypes; and what are the consequential implications for fairness? Through a series of experiments, we delve into these questions and show that LLMs tend to inherit social biases from their training data which significantly impact their fairness in tabular classification tasks. Furthermore, our investigations show that in the context of bias mitigation, though in-context learning and finetuning have a moderate effect, the fairness metric gap between different subgroups is still larger than that in traditional machine learning models, such as Random Forest and shallow Neural Networks. This observation emphasizes that the social biases are inherent within the LLMs themselves and inherited from their pretraining corpus, not only from the downstream task datasets. Besides, we demonstrate that label-flipping of in-context examples can significantly reduce biases, further highlighting the presence of inherent bias within LLMs.
BibTeX
@inproceedings{liu2024confronting,
  author = {Yanchen Liu and Srishti Gautam and Jiaqi Ma and Himabindu Lakkaraju},
  title = {{Confronting LLMs with Traditional ML: Rethinking the Fairness of Large Language Models in Tabular Classifications}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.198},
}
Figure