From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models

Julia Mendelsohn, Ronan Le Bras, Yejin Choi, Maarten Sap

Abstract

Dogwhistles are coded expressions that simultaneously convey one meaning to a broad audience and a second one, often hateful or provocative, to a narrow in-group; they are deployed to evade both political repercussions and algorithmic content moderation. For example, in the sentence 'we need to end the cosmopolitan experiment,' the word 'cosmopolitan' likely means 'worldly' to many, but secretly means 'Jewish' to a select few. We present the first large-scale computational investigation of dogwhistles. We develop a typology of dogwhistles, curate the largest-to-date glossary of over 300 dogwhistles with rich contextual information and examples, and analyze their usage in historical U.S. politicians' speeches. We then assess whether a large language model (GPT-3) can identify dogwhistles and their meanings, and find that GPT-3's performance varies widely across types of dogwhistles and targeted groups. Finally, we show that harmful content containing dogwhistles avoids toxicity detection, highlighting online risks of such coded language. This work sheds light on the theoretical and applied importance of dogwhistles in both NLP and computational social science, and provides resources for future research in modeling dogwhistles and mitigating their online harms.

Introduction: The Challenge of Plausible Deniability

In political and social discourse, "dogwhistles" serve as a strategic tool for maintaining plausible deniability. Unlike overt slurs, dogwhistles are intentionally designed to appear benign to the general public while signaling a specific, often exclusionary or hateful, message to a target in-group. This dual-meaning structure creates a significant hurdle for Natural Language Processing (NLP) systems. Most contemporary hate-speech detection models are trained to identify explicit toxicity; because dogwhistles are "clean" on the surface, they effectively bypass automated moderation filters.

The mechanism of a dogwhistle: a single message conveys a literal meaning to the outgroup while signaling a coded persona and meaning to the ingroup, maintaining plausible deniability.

Methodology: Taxonomy and Glossary Development

To address this gap, researchers developed a structured framework to categorize dogwhistles based on three primary dimensions:

  1. Register: Distinguishing between formal/offline political rhetoric and informal/online internet slang.
  2. Type: Categorizing whether a term merely signals a Persona (Type I) or adds a specific Hidden Meaning (Type II).
  3. Persona: Identifying the specific in-group identity associated with the term (e.g., "white supremacist," "anti-vax," or "transphobic").

The study produced the Dogwhistle Glossary, a curated collection of 340 English-language dogwhistles and over 1,000 surface forms, complete with contextual examples. This resource allows for the quantitative study of language that was previously only analyzed through qualitative social science lenses.

Historical Evolution in U.S. Politics

By applying this glossary to the U.S. Congressional Record (1920–2020), the researchers tracked the diachronic shift of coded rhetoric. Using DW-NOMINATE scores—a standard metric for political ideology—the study observed how certain terms migrated between liberal and conservative circles.

A key finding was the sharp increase in racial dogwhistles following the 1954 Brown v. Board of Education decision. This trend aligns with the "Southern Strategy," where explicit racism was replaced by coded appeals to maintain political viability while signaling to segregationist bases.

Historical trends in the usage of dogwhistles within the U.S. Congressional Record (1920–2020), showing significant spikes following major civil rights milestones and political shifts.

Probing Large Language Models

The researchers tested GPT-3’s ability to decode these terms. The results revealed that LLMs are "formal students": GPT-3 identified 69.4% of formal political dogwhistles (like "family values") but only 12.9% of informal, internet-native ones (such as specific emojis or alt-right slang).

Interestingly, context proved vital. GPT-3’s decoding accuracy improved five-fold when the prompt explicitly stated the term was being used "secretly." This suggests that while the model has "learned" these associations from its training data, it requires specific triggers to access and surface that knowledge. However, performance was inconsistent across groups, with the model struggling significantly more with transphobic dogwhistles (22.9% accuracy) than with homophobic (73.7%) or Islamophobic (65.4%) ones.

The Toxicity Detection Gap

The most critical finding for online safety is the "moderation gap." Using HateCheck templates, the researchers demonstrated that swapping a standard group label (e.g., "Jewish") for a dogwhistle (e.g., "cosmopolitan") caused toxicity scores from Google’s Perspective API to drop drastically. In many cases, the scores fell below the threshold required for automated flagging. This confirms that dogwhistles function as a highly effective "cloaking device" against current safety tools, allowing harmful content to proliferate unchecked.

Conclusion and Future Directions

This research highlights that dogwhistles are a "moving target." As a term becomes widely recognized by the out-group (becoming "too loud"), it loses its utility for plausible deniability and is often replaced by new, more obscure rhetoric. Future work must move toward automatic discovery to identify these emerging terms in real-time. While the creation of such a glossary carries a dual-use risk—potentially helping bad actors refine their evasion tactics—the authors argue that the benefit to researchers and moderators in building more robust, context-aware safety systems is paramount.

Summary updated

Abstract

Dogwhistles are coded expressions that simultaneously convey one meaning to a broad audience and a second one, often hateful or provocative, to a narrow in-group; they are deployed to evade both political repercussions and algorithmic content moderation. For example, in the sentence 'we need to end the cosmopolitan experiment,' the word 'cosmopolitan' likely means 'worldly' to many, but secretly means 'Jewish' to a select few. We present the first large-scale computational investigation of dogwhistles. We develop a typology of dogwhistles, curate the largest-to-date glossary of over 300 dogwhistles with rich contextual information and examples, and analyze their usage in historical U.S. politicians' speeches. We then assess whether a large language model (GPT-3) can identify dogwhistles and their meanings, and find that GPT-3's performance varies widely across types of dogwhistles and targeted groups. Finally, we show that harmful content containing dogwhistles avoids toxicity detection, highlighting online risks of such coded language. This work sheds light on the theoretical and applied importance of dogwhistles in both NLP and computational social science, and provides resources for future research in modeling dogwhistles and mitigating their online harms.
BibTeX
@inproceedings{mendelsohn2023from,
  author = {Julia Mendelsohn and Ronan Le Bras and Yejin Choi and Maarten Sap},
  title = {{From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.845},
}
Figure