Metamorphic Testing and Certified Mitigation of Fairness Violations in NLP Models

Pingchuan Ma, Shuai Wang, Jin Liu

Introduction

As Natural Language Processing (NLP) models are increasingly deployed in high-stakes domains such as hiring, credit scoring, and legal assessments, the risk of algorithmic bias has become a critical concern. A model might inadvertently associate specific genders, races, or nationalities with negative sentiments or lower scores, leading to systemic unfairness.

Examples of fairness violations where changing a sensitive attribute (gender or nationality) alters the model's sentiment prediction.

Identifying these biases is challenging because natural language is unstructured. Traditional testing often relies on manual, hand-coded rules that fail to capture the complexity of human identity. Furthermore, most existing "de-biasing" techniques require access to internal model weights or expensive retraining, which is impossible for users of proprietary "black-box" commercial APIs. To address these challenges, researchers developed MT-NLP, a framework for automated bias detection and certified mitigation.

Methodology

The MT-NLP framework consists of a two-stage pipeline: automated detection via metamorphic testing and certified mitigation through a "neutral layer."

Metamorphic Testing (MT) for Detection

Metamorphic testing evaluates whether a model adheres to a "Metamorphic Relation" (MR). In this context, the MR dictates that changing a sensitive attribute (e.g., gender) in a sentence should not alter the model's prediction. The framework automates this process using:

  • Knowledge Graphs: Leveraging ConceptNet to identify human-related nouns and their relationships.
  • Analogy Mutation: Using word embeddings to find analogous terms (e.g., mapping "actor" to "actress" via vector arithmetic).
  • Active Mutation: Inserting sensitive adjectives (e.g., "male," "Chinese") before neutral nouns to probe for output changes.
  • Fluency Filtering: Applying language fluency metrics to ensure generated test sentences remain realistic.

Certified Mitigation via the Neutral Layer

To rectify identified violations without retraining, the authors introduce a Neutral Layer. Instead of relying on a single prediction for an input $x$, the system generates $k$ perturbed versions (mutations).

The Neutral Layer architecture: an input sample and its mutations are processed by the model, and their outputs are weighted to produce a "smoothed" fair prediction.

The final "smoothed" output $\hat{y}$ is a weighted average of the original and mutated predictions. This approach provides $(\epsilon, k)$-fairness, a mathematical guarantee that the model's output remains stable across mutations within a parameter $\epsilon$.

Experimental Results

The framework was evaluated using 20,000 sentences from the IMDB Large Movie Review Dataset. Testing across five models—including commercial APIs from Amazon, Google, and Microsoft—revealed 2,874 discriminatory inputs.

NLP Model/Service # of Tested Sentences # of Fairness Violations
Amazon Comprehend API 28,425 197
Google NLP API 30,504 140
Microsoft NLP API 28,721 104
CNN (Local) 174,183 1,111
Logistic Regression (LR) 174,183 1,322

Key findings include:

  • Mitigation Effectiveness: At a low fairness parameter ($\epsilon = 0.1$), the mitigation technique fixed over 98% of violation cases on average.
  • Black-Box Versatility: The "neutral layer" successfully provided fairness guarantees for proprietary APIs without requiring internal access.
  • Keyword Limitations: Commercial APIs that use simple keyword filtering were still found susceptible to noun-based swaps (e.g., "uncle" vs. "aunt"), highlighting the need for more robust detection methods like MT-NLP.

Conclusion and Future Work

MT-NLP demonstrates that systematic exploration of subpopulation groups through automated perturbation is a powerful tool for auditing and securing NLP models. While the mitigation adds some computational overhead, its parallelizable nature makes it practical for real-world deployment. Future research will focus on expanding the scope of sensitive attributes beyond gender to include religion and sexual orientation, as well as reducing the dependency on external knowledge graphs.

Summary updated

BibTeX
@inproceedings{ma2020metamorphic,
  author = {Pingchuan Ma and Shuai Wang and Jin Liu},
  title = {{Metamorphic Testing and Certified Mitigation of Fairness Violations in NLP Models}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/64},
}
Figure