TL;DR
This paper introduces a framework and a checklist (HARMCHECK) for responsibly handling and presenting harmful text in NLP research. It addresses the critical need for ethical guidelines to mitigate risks to data subjects, researchers, and readers, moving beyond just technical solutions to encompass the entire research lifecycle.
What Problem Does This Paper Address?
The NLP community increasingly deals with harmful text (e.g., hate speech, misinformation, stereotypes) found in datasets. While the risks of deploying models trained on such data are gaining attention, the ethical challenges of handling and presenting this harmful content during the research process itself are less understood. This includes how to ethically source, process, analyze, and report on data that can cause emotional, psychological, or societal harm. The paper aims to fill this gap by providing a structured approach to identify, categorize, and mitigate these harms throughout the research lifecycle, from data curation to publication.
Methods
The paper proposes an analytical framework that categorizes harms along three axes:
- Harm Type: What kind of harm is it (e.g., misinformation, hate speech, stereotypes)?
- Harm Encounter: Is the harmful content "sought" (intentionally collected for study) or "unsought" (encountered incidentally)?
- Who is Affected: Which groups are at risk (data subjects, data handlers/researchers, readers/reviewers)?
Based on this framework, they introduce HARMCHECK, a checklist designed to guide researchers in transparent, responsible, and reflective reporting of harmful text. HARMCHECK covers aspects like risk assessment, content warnings, distancing authors from harmful content, and respecting data subjects. The paper also draws inspiration from journalistic practices like "preview," "distance," and "disclaim" for presenting sensitive material.
Datasets
The paper does not introduce new datasets. Instead, it discusses the use of existing large-scale, internet-scraped datasets like Common Crawl and WebText, which are common sources of harmful content. It also references datasets used in prior work for specific types of harmful content, such as:
- HATEMOJICHECK (Kirk et al., 2022) for emoji-based hate.
- Datasets for misinformation analysis (Zubiaga et al., 2016).
- Multimodal datasets containing negative stereotypes (Birhane et al., 2021).
Key Insights and Ideas
- Harm is Contextual and Multifaceted: The impact of harmful text isn't inherent to the text itself but depends heavily on context, historical setting, and who encounters it. This makes defining and measuring harm complex.
- "Sought" vs. "Unsought" Harm: The paper distinguishes between intentionally studying harmful content ("sought") and encountering it incidentally ("unsought"). While "sought" harm might involve more awareness, "unsought" harm can be insidious due to its unexpected nature.
- Beyond Technical Fixes: Ethical handling of harmful text requires more than just algorithmic solutions; it necessitates a shift in research practices, transparency, and a critical examination of societal norms embedded in data.
- Shared Responsibility: Harm affects multiple stakeholders: data subjects (through representation or erasure), researchers and annotators (through vicarious trauma), and readers/reviewers (through distress). Protocols must consider all these groups.
- HARMCHECK as a Starting Point: The proposed checklist is not a rigid compliance tool but a facilitator for ongoing dialogue and reflection on responsible research practices.
Key Results
The paper doesn't present quantitative results from experiments. Instead, its key contributions are:
- An Analytical Framework: A structured way to understand and categorize harms related to text data in NLP research.
- HARMCHECK Checklist: A practical tool to guide researchers in documenting and mitigating risks associated with handling and presenting harmful text.
- Case Studies: Demonstrations of how HARMCHECK can be applied to prior work on negative stereotypes, emoji-based hate, and misinformation, highlighting specific steps taken or recommended.
- Guidelines for Practice: Concrete advice for researchers on mitigating harm during data curation, research execution, and publication.
Limitations and Future Work
- Contextual Focus: The work is primarily focused on NLP research settings. The authors acknowledge that other text-processing contexts might have different practices and norms.
- "Open Class" Problem: Harms and harmful text are an "open class" problem, meaning it's impossible to enumerate all possible types. The paper focuses on harms arising during dataset curation, research, and publication, not downstream harms from model deployment.
- "Open Class" People: What constitutes harm varies significantly between individuals. The recommendations are based on established cases of demonstrable harm but are not exhaustive.
- Researcher Positionality: The authors acknowledge their positionality as well-resourced researchers from Western institutions, which may lead to blind spots, particularly concerning the perspectives of data labellers and annotators. They encourage treating these individuals as integral team members.
- Future Work: The authors hope their work initiates a broader conversation and encourages further development of responsible, safe, and ethical research practices in NLP. They suggest that more research is needed on how to effectively support data labellers and annotators.
Summary updated