Introduction
Large-scale pre-trained language models (PLMs) such as BERT and RoBERTa are known to inherit societal biases from their training corpora. While research has extensively documented biases related to gender, race, and religion, sentiment bias—the latent association of specific words with positive or negative polarity—remains less explored at the token level. When a model internally assigns sentiment to a neutral word (e.g., "systematic" or "speculation"), it can inadvertently skew downstream tasks like automated grading or market analysis. This paper addresses the challenge of detecting these hidden biases using novel prompt-based probing techniques.
Methodology
To extract latent sentiment associations, the researchers developed two distinct methods: the Sentiment Association Test (SAT) and the Sentiment Shift Test (SST).
Sentiment Association Test (SAT)
The SAT measures the strength of association between a target word and positive versus negative contexts. By appending a prompt like "It was [MASK]." to a review (e.g., "The movie was awesome"), the test evaluates whether the model is more likely to predict a specific target word in the masked position for positive reviews compared to negative ones.
Sentiment Shift Test (SST)
The SST employs a perturbation-based approach to observe how a "neutral" word influences the model's overall sentiment prediction. In this test, a neutral word is appended $K$ times to a review. The researchers then measure the shift in the model's sentiment prediction for that review using a new metric, $q$, to quantify the strength of the bias. If the repeated addition of a word like "religious" causes a positive review to be classified as negative, that word is identified as possessing a negative sentiment bias.
Experimental Setup
The study utilized human-annotated lexicons, including the VADER lexicon for polar words and the MPQA opinion corpus for neutral words. To provide context, 4,000 reviews were sampled from IMDB, Amazon, Yelp, and SST-2. The primary models evaluated were RoBERTa-base and RoBERTa-large.
Key Findings
The "Neutral" Illusion
The results indicate that human perception of neutrality does not always align with model representations. Approximately 39.25% of the 400 words labeled as "neutral" in human lexicons showed statistically significant sentiment bias. Words such as "vanilla," "cheese," and "speculation" were frequently treated as negative by the models, while "shone," "dominant," and "clout" exhibited positive bias.
Fine-tuning as an Amplifier
A critical finding is that fine-tuning PLMs on sentiment classification tasks significantly amplifies existing biases. In RoBERTa-base, the number of biased words increased by over 124% following fine-tuning. This suggests that the process of adapting a model to a specific task can reinforce and expand latent token-level associations.
Consistency and Model Scale
The SAT and SST methods showed high agreement, particularly for positively biased words, indicating that these biases are stable features of the model's architecture. While larger models like RoBERTa-large exhibited different specific bias patterns, they remained equally susceptible to the amplification effects of fine-tuning.
Conclusion
This research highlights the prevalence of token-level sentiment bias in PLMs and the risks associated with standard fine-tuning procedures. Future work should focus on expanding these tests to a broader range of architectures, such as GPT-4, and developing robust de-biasing techniques to ensure that "neutral" words remain truly neutral in automated systems.
Summary updated