TestSGD: Interpretable Testing of Neural Networks against Subtle Group Discrimination.

Mengdi Zhang, Jun Sun, Jingyi Wang, Bing Sun

Introduction

As neural networks are increasingly deployed in high-stakes domains like criminal justice, hiring, and credit scoring, ensuring their fairness has become a critical priority. However, traditional fairness testing often falls short by focusing on broad categories such as "Gender" or "Race" in isolation. This approach overlooks subtle group discrimination—biases that emerge at the intersection of multiple features, such as "older Hispanic males" or "divorced women in specific zip codes."

Identifying these subgroups is a classic "needle in a haystack" problem due to the astronomical number of possible feature combinations. To address this, researchers have developed TESTSGD, a black-box testing framework designed to automatically uncover these hidden biases, provide interpretable explanations for them, and offer a pathway for mitigation.

The Challenge of Subtle Discrimination

Most existing fairness metrics evaluate either individual discrimination (ensuring similar individuals receive similar outcomes) or global group fairness (checking for parity across a single sensitive attribute). Subtle discrimination is more insidious because a model can appear fair on a global scale while harboring significant bias against specific, smaller subgroups. Because neural networks are often "black boxes," these intersectional biases can remain hidden until they cause real-world harm. TESTSGD aims to make these biases visible and actionable for human auditors.

The TESTSGD Framework

TESTSGD operates through a systematic three-phase process that balances computational efficiency with statistical rigor.

The three-phase workflow of TESTSGD: Rule Set Generation, Group Fairness Identification, and Mitigation.

Phase 1: Candidate Rule Set Generation

Rather than searching the entire space of feature combinations, TESTSGD mines "frequent rule sets." These are combinations of sensitive features that represent a significant portion of the population (defined by a support threshold). These rules are formatted to be human-interpretable, using linear inequalities for continuous data (e.g., age > 40) and set membership for categorical data.

Phase 2: Group Fairness Identification

For every subgroup identified in Phase 1, the system calculates a fairness score. Since it is computationally impossible to test every possible input, TESTSGD employs a sampling-based approach. It generates synthetic samples to estimate the model's behavior for the subgroup and its complement. Crucially, it provides a theoretical error bound, ensuring that the reported fairness score is statistically accurate within a 95% confidence interval.

Phase 3: Discrimination Mitigation

Once a biased subgroup is identified, the framework can automatically generate synthetic training data. These new samples satisfy the "unfair" rules but are assigned "favorable" labels. By retraining the model on this augmented dataset, the network can "unlearn" the specific bias without requiring a complete architectural overhaul.

Experimental Setup and Datasets

The effectiveness of TESTSGD was evaluated across eight diverse datasets, spanning both structured and natural language data:

  • Structured Data: Census Income, Bank Marketing, German Credit, COMPAS (recidivism), Crime (community violence), and Law School (bar exam passage).
  • Text Data: Wikipedia Talk Pages (toxicity detection) and IMDB (sentiment analysis).

In the text-based datasets, sensitive features were identified by the presence of specific identity terms, such as "gay," "protestant," or "white," allowing the framework to detect biases in NLP models that might otherwise go unnoticed.

Key Results and Insights

The researchers found that subtle bias is pervasive, even in models that might pass standard fairness checks. TESTSGD successfully identified significant fairness gaps across all tested models.

Top biased subgroups and their corresponding fairness scores across various datasets.

Major Findings:

  • Extreme Intersectional Bias: In the COMPAS dataset, TESTSGD uncovered a subgroup (male, age ≥ 40, Hispanic or other race) with a massive 62.4% fairness gap. Similarly, in the Crime dataset, the model was 60.7% more likely to predict high crime rates for communities characterized by low Caucasian populations and high rates of divorced females.
  • Efficiency and Scalability: For most structured datasets, the framework identified the top biased subgroups in under 16 minutes. While high-dimensional datasets like Crime and text-based datasets took longer (up to 19 hours), the process remains practical for pre-deployment auditing.
  • Effective Mitigation: The authors demonstrated that targeted retraining works. By focusing on the specific "subtle" groups identified, they reduced the bias in the COMPAS model from 57.7% to just 4.2% with negligible impact on the model's overall accuracy.

Conclusion and Future Work

TESTSGD represents a significant step forward in making AI fairness testing both interpretable and actionable. By moving beyond global metrics and focusing on the "subtle" intersections of identity, it provides developers with the tools needed to audit and fix models before they are deployed.

Future work will likely focus on refining the support threshold (θ) to better capture biases in very small populations without exploding the search space, and improving the sampling methods to ensure synthetic data remains within realistic distributions. Ultimately, while TESTSGD provides the statistical evidence, it remains a tool for human experts to decide which disparities constitute unjustified discrimination in a real-world context.

Summary updated

BibTeX
@article{zhang2023testsgd,
  author = {Mengdi Zhang and Jun Sun and Jingyi Wang and Bing Sun},
  title = {{TestSGD: Interpretable Testing of Neural Networks against Subtle Group Discrimination.}},
  journal = {ACM Trans. Softw. Eng. Methodol.},
  year = {2023},
  doi = {10.1145/3591869},
}
Figure