The State of Profanity Obfuscation in Natural Language Processing

Debora Nozza, Dirk HovyarXiv 2022 · DOI · arXiv

Introduction

In the rapidly evolving field of Natural Language Processing (NLP), particularly within hate speech detection and toxic content analysis, researchers frequently encounter a significant ethical and technical challenge: how to report offensive language in scientific publications. Researchers from Bocconi University recently addressed this "messy corner" of the field, investigating the inconsistencies and biases inherent in current profanity obfuscation practices.

The Reporting Conundrum

When documenting hate speech, authors face a "conundrum of reporting." On one hand, printing slurs in full can be harmful to readers and contributes to the proliferation of hate speech online as these papers are indexed by search engines. On the other hand, heavy masking (e.g., ****) can render data unintelligible, hindering reproducibility and accessibility. This is especially problematic for non-native English speakers who may lack the cultural context to decipher heavily obfuscated slang.

Examples of profanity obfuscation found in recent NLP literature.

As shown in the examples above, researchers often resort to various masking techniques, such as replacing vowels with asterisks. However, the lack of a unified standard leads to significant variation across the literature.

Methodology: Surveying the Field

To quantify the current state of the field, the authors conducted a systematic survey of 150 papers from major 2021 NLP venues, including ACL, EACL, EMNLP, and the Workshop on Online Abuse and Harms (WOAH). They tracked the frequency of obfuscation, the specific symbols used, and the presence of content warnings.

Distribution of obfuscated profanities across major 2021 NLP conference proceedings.

The survey revealed that obfuscation is widespread but unevenly distributed across venues. Furthermore, the study found that fewer than 20% of surveyed papers included formal disclaimers or content warnings, despite the sensitive nature of the content.

The PROF Framework

In response to these findings, the researchers introduced PROF, a multilingual resource and Python library designed to standardize profanity handling. PROF moves beyond simple "bad word lists" by mapping terms to BabelNet synset IDs. This semantic mapping allows researchers to link offensive concepts across different languages programmatically.

The initial PROF resource includes 203 profanities across five languages:

  • English (50)
  • French (44)
  • German (19)
  • Italian (42)
  • Spanish (48)

The accompanying Python package, profanity-obfuscation, enables automated "first-vowel obfuscation" (e.g., f*ck) and provides a mechanism to "reveal" original terms for researchers who need the full context for data analysis.

Key Insights and Bias

A critical finding of the study was the "English-centric" bias in obfuscation. While English profanities were frequently masked, offensive terms in other languages were often left in their raw form. This suggests that authors and reviewers may be less sensitive to, or less aware of, profanity in non-English contexts.

Furthermore, the authors argue that scientific reporting serves a "meta-function"—talking about language rather than using it to attack. Effective obfuscation should preserve this distinction, maintaining the data's utility for research while minimizing potential harm.

Conclusion and Future Directions

The PROF framework represents a significant step toward standardizing how the NLP community handles sensitive data. While currently limited to five languages and 203 terms, the authors intend for PROF to be a community-driven resource. Future work will focus on expanding language coverage, addressing the rapid evolution of online slurs, and tackling the unique challenges of profanity in multimodal contexts, such as text within images.

By providing a programmatic, multilingual standard, the researchers aim to improve the reproducibility and accessibility of hate speech research while fostering a more inclusive and ethical scientific environment.

Summary updated

Abstract

Work on hate speech has made the consideration of rude and harmful examples in scientific publications inevitable. This raises various problems, such as whether or not to obscure profanities. While science must accurately disclose what it does, the unwarranted spread of hate speech is harmful to readers, and increases its internet frequency. While maintaining publications' professional appearance, obfuscating profanities makes it challenging to evaluate the content, especially for non-native speakers. Surveying 150 ACL papers, we discovered that obfuscation is usually employed for English but not other languages, and even so quite uneven. We discuss the problems with obfuscation and suggest a multilingual community resource called PrOf that has a Python module to standardize profanity obfuscation processes. We believe PrOf can help scientific publication policies to make hate speech work accessible and comparable, irrespective of language.
BibTeX
@misc{nozza2022state,
  author = {Debora Nozza and Dirk Hovy},
  title = {{The State of Profanity Obfuscation in Natural Language Processing}},
  year = {2022},
  eprint = {2210.07595},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2210.07595},
}
Figure