TL;DR
While Large Language Models (LLMs) are increasingly being adapted for tabular classification tasks, this paper reveals that they inherit severe social biases from their pre-training data, often underperforming traditional machine learning models like Random Forests in fairness metrics. Standard mitigation techniques like data resampling fail to help LLMs, though full fine-tuning shows promise in leveling the fairness playing field.
What Problem Does This Paper Address?
Tabular data is the backbone of high-stakes decision-making systems, from predicting credit risks to evaluating criminal recidivism. Recently, a trend has emerged to serialize tabular data into natural language prompts so that powerful LLMs can perform classification tasks.
However, LLMs are trained on massive internet text corpora rife with human stereotypes. This paper addresses a critical safety gap: Do LLMs import these harmful social biases into tabular classification? If so, how does their fairness compare to traditional machine learning (ML) baselines, and can standard bias-mitigation techniques save them?
Methods
The authors evaluate GPT-3.5 (gpt-3.5-turbo) against two traditional baselines: a Random Forest (RF) and a shallow 3-layer Neural Network (NN).
To feed tabular data into the LLM, they serialize rows into text strings (e.g., "workclass: Private, hours per week: 40, sex: Female..."). They test the LLM across several regimes:
- Regular Zero-Shot Prompting: Asking the model for a prediction based purely on the serialized row and task description.
- In-Context Learning (ICL): Providing 50 few-shot examples within the prompt before the test instance.
- Label-Flipped ICL: Intentionally flipping the target labels of the 50 few-shot examples to see if the model relies on input-label mappings or internal semantic priors.
- Fine-Tuning (FT): Training the model on the full dataset via OpenAI's API, including variants using data resampling (oversampling the minority group or undersampling the majority group).
Fairness is evaluated using standard metrics: Statistical Parity (SP) and Equality of Opportunity (EoO).
Datasets
The study utilizes three widely recognized tabular benchmarks for fairness research:
- Adult Income Dataset: Predicts whether an individual earns >$50k/year. Protected attribute: Sex (Female as the disadvantaged group).
- German Credit Dataset: Classifies bank customers as good or bad credit risks. Protected attribute: Sex (Female as the marginalized group).
- COMPAS Dataset: Predicts two-year criminal recidivism. Protected attribute: Race (African-American as the disadvantaged group).
Key Insights and Ideas
- Biases are Inherent, Not Just Downstream: The unfairness observed in LLMs isn't just a reflection of imbalances in the tabular datasets; it stems directly from the model's pre-training corpus. Even when protected attributes (like race or sex) are completely removed from the prompt, a significant fairness gap remains.
- The Label-Flipping Paradox: Flipping the labels of few-shot examples drastically reduces the fairness gap, nearly eliminating the bias in some cases. This proves that the LLM's tabular predictions are heavily driven by its internal semantic stereotypes rather than purely learning data correlations. However, this fairness gain comes at the cost of overall predictive accuracy.
- Resampling Fails for LLMs: In traditional ML, oversampling or undersampling data is a reliable way to balance group representations and improve fairness. For LLMs, resampled fine-tuning yields highly inconsistent results, demonstrating that LLMs process data distributions fundamentally differently than traditional architectures.
Key Results
In zero-shot settings, GPT-3.5 exhibits massive fairness disparities that frequently dwarf traditional ML models. For instance, on the Adult dataset, the Equality of Opportunity (EoO) gap between men and women reached 0.483 for GPT-3.5, compared to just 0.195 for Random Forest.
While regular few-shot learning (ICL) narrows this gap slightly, the disparities remain higher than traditional baselines. However, Regular Fine-Tuning (FT) proved highly effective, dropping the Adult EoO gap to 0.0714—successfully outperforming the Neural Network baseline (0.123).
Limitations and Future Work
The authors acknowledge two primary limitations:
- Model Scope: The study focuses exclusively on GPT-3.5. Other state-of-the-art models (like GPT-4, Claude, or open-source alternatives like Llama) might exhibit different bias profiles.
- Prompt Dependency: The experiments rely on a single serialization template style. The impact of advanced prompting techniques, such as Chain-of-Thought (CoT), remains unexplored.
Future work will expand the evaluation to a broader suite of LLMs and investigate how diverse prompting strategies alter bias propagation in tabular domains.
Summary updated