TL;DR
This paper introduces "aporophobia," a social bias against the poor, as a significant and overlooked form of toxic language. The authors demonstrate its prevalence on social media and argue that current NLP datasets and models are inadequate for its detection, highlighting the urgent need for specialized resources and methods.
What Problem Does This Paper Address?
While NLP research has extensively focused on detecting toxic language targeting various groups like women and ethnic minorities, prejudice against the poor, termed "aporophobia," has largely been ignored. This oversight is problematic because aporophobia can be psychologically damaging to individuals, hinder poverty reduction efforts, and is argued by some to be even more prevalent than other forms of discrimination. The paper aims to bring attention to this issue, characterize aporophobia, demonstrate its presence online, and highlight the inadequacy of existing NLP tools to address it.
Methods
The researchers employed a multi-pronged approach:
- Literature Review: They synthesized social science literature to define and characterize aporophobia, establishing its societal impact.
- Social Media Analysis (Twitter):
- Tweet Collection: They collected tweets related to "poor people" and "rich people" using carefully curated query terms.
- Word Association Analysis: Using Pointwise Mutual Information (PMI), they identified words strongly associated with "poor people" but rarely with "rich people." This helped reveal common themes and stereotypes.
- Topic Modeling: They used BERTopic to identify prevalent themes and topics discussed in tweets mentioning "poor people," uncovering both explicit and implicit expressions of aporophobia.
- Dataset Evaluation: They systematically examined nine widely used English-language toxicity datasets to assess their suitability for detecting aporophobia. This involved checking if the datasets included "poor" or related terms and if these instances were labeled as toxic.
- Model Performance Evaluation: They evaluated the performance of three pre-trained, publicly available toxicity prediction models on aporophobic instances, both those labeled as "toxic" and "non-toxic" by human annotators.
Datasets
The paper discusses the following datasets:
- Existing Toxicity Datasets:
- Civil Comments Dataset
- Wiki Toxicity
- Abusive and Hateful Tweet Corpus
- Social Bias Inference Corpus (SBIC)
- Unhealthy Comments Corpus
- Hate Speech and Offensive Language Tweet Corpus
- Contextual Abuse Dataset (CAD)
- Offensive Language Identification Dataset (OLID)
- Hate Speech and Offensive Content Identification in Indo-European Languages (HASOC-2019)
- Newly Introduced Benchmarks: The paper does not introduce new datasets or benchmarks but rather evaluates existing ones for their suitability in detecting aporophobia.
Key Insights and Ideas
- Aporophobia is Pervasive and Subtle: The study reveals that aporophobic attitudes are not only present but often expressed subtly, making them difficult for both human annotators and current NLP models to detect, especially when instances are labeled as "non-toxic."
- Existing Datasets are Ill-Suited: Despite the prevalence of aporophobia, existing toxicity datasets are largely inadequate. They either lack sufficient examples of aporophobia or fail to label subtle instances as toxic, hindering the training of effective detection models.
- Stereotypes are Deeply Ingrained: The word association analysis and topic modeling highlight common negative stereotypes linked to poverty, such as associations with drug abuse, mental illness, and unsanitary conditions.
- Counter-Speech Exists: While aporophobic views are present, the analysis also identified instances of counter-speech, where users actively dispute stereotypes and discriminatory actions against the poor. This suggests a societal awareness and pushback against such biases.
- Impact on Policy: The paper emphasizes that aporophobia can directly impede poverty reduction efforts by fostering a societal belief that the poor are responsible for their own plight and thus undeserving of help.
Key Results
- Twitter Analysis: The study found a significant presence of aporophobic views expressed on Twitter. Word association revealed terms like "addicts," "mental," "drugs," and "crime" strongly linked to "poor people." Topic modeling identified themes related to homelessness, lack of housing, and negative portrayals of the poor.
- Dataset Evaluation: Across nine major toxicity datasets, the number of instances mentioning "poor" and labeled as "toxic" was very low, often in the dozens. The Civil Comments dataset, being the largest and unfiltered, had the most instances mentioning "poor" (over 19K), but over 95% were labeled as non-toxic.
- Model Performance: When tested on aporophobic instances, existing toxicity models showed moderate to high recall on instances already labeled as "toxic" but failed to recognize aporophobia in more subtle, "non-toxic" instances. For example, models achieved 0% recall on aporophobic instances originally labeled as "non-toxic."
Limitations and Future Work
- Limitations:
- The study focused primarily on English-language resources and the Twitter platform, which has a biased demographic representation (predominantly US users). This means the findings may not generalize to other languages, regions, or platforms.
- The study excluded derogatory terms and slurs explicitly associated with the poor, as these are generally easier to detect. However, these explicit expressions are still important for a comprehensive understanding.
- The study focused only on textual data, neglecting multimodal content like images and videos.
- Future Work:
- Develop specialized resources (datasets, lexicons, models) for detecting and countering aporophobia.
- Design novel socio-technical mechanisms to address this bias.
- Explore alternative data collection techniques beyond random sampling to efficiently gather aporophobic content.
- Develop practical annotation guidelines and training programs for annotators to improve their understanding and recognition of aporophobia.
- Create an "aporophobia index" by tracking online expressions and actions to inform policy and public awareness.
- Expand research to include other languages, world regions, and cultural contexts.
- Investigate multimodal data for a more comprehensive analysis.
Summary updated