Abstract
Sentiment analysis (SA) systems, though widely applied, have been demonstrated to produce biased results. While existing research focuses on generating test cases to reveal unfairness before deployment, there is a lack of tools for monitoring biased predictions at runtime. This paper introduces BiasRV, the first tool designed to raise alarms when a deployed SA system makes a biased prediction on live input. By dynamically generating gender-discriminatory mutants and applying a high-efficiency two-step heuristic, BiasRV detects biased predictions with significantly reduced computational overhead.
The Challenge of Runtime Fairness
Sentiment analysis models often harbor hidden biases; for instance, a model might label a movie review as "positive" when the protagonist is named "James" but "negative" when the name is changed to "Julissa." While researchers have developed tools to uncover these biases before deployment using pre-defined templates, these methods struggle with the unpredictable nature of real-world user queries. BiasRV addresses this gap by providing a runtime "guardrail" that ensures individual predictions remain fair once a model is live.
BiasRV Architecture and Methodology
BiasRV operates as a wrapper around existing SA systems, treating them as black boxes. The system follows a multi-stage process to verify the fairness of each incoming query.
- Dynamic Template Extraction: BiasRV uses Natural Language Processing (NLP) techniques, such as Named Entity Recognition (NER) and Coreference Resolution, to identify "protected tokens" (names and pronouns) in a live query. It then creates a template by replacing these tokens with placeholders.
- Mutant Generation: Using the extracted template, the tool generates "mutants"—alternative versions of the text that differ only in gender information. For example, it might generate 30 versions with female names and 30 with male names.
- Distributional Fairness Verification: A prediction is considered fair if the proportion of positive sentiment results for male mutants is nearly equal to the proportion for female mutants (within a threshold $\alpha$, typically 0.10).
Efficiency through Heuristics
Running dozens of extra model inferences for every user query is computationally expensive. To make BiasRV practical for production environments, the authors implemented a two-step heuristic:
- Step 1 (Sampling): The system samples a small number (e.g., 4) of mutants from each gender. If the SA system predicts the same sentiment for all of them, it is assumed to be unbiased.
- Step 2 (Full Check): If there is a conflict in the initial sample, BiasRV performs a full check of all generated mutants to calculate the distributional fairness property.
Experimental Results
The researchers evaluated BiasRV using 25,000 IMDB movie reviews and a fine-tuned BERT model. Key findings include:
- Overhead Reduction: The two-step heuristic reduced the computational overhead of mutant analysis by 73.81% compared to exhaustive checking.
- Detection Accuracy: Despite the optimization, the heuristic only missed 6.7% of biased predictions.
- Real-world Utility: In the IMDB test set, BiasRV identified 15 specific instances of clear biased predictions that would have otherwise gone unnoticed.
Limitations and Future Work
BiasRV currently relies on the accuracy of underlying NLP components for NER and coreference resolution. Future work aims to expand the tool's scope beyond binary gender to include race, religion, and occupation-based biases, as well as further optimizing the mutation engine to reduce initial template generation time.
Summary updated