On the Role of Speech Data in Reducing Toxicity Detection Bias

Samuel J. Bell, Mariano Coria Meglioli, Megan Richards, Eduardo Sánchez, Christophe Ropers, Skyler Wang, Adina Williams, Levent Sagun, Marta R. Costa-jussà

TL;DR

Text-based toxicity detectors often flag benign sentences simply because they mention marginalized demographic groups. This paper demonstrates that incorporating raw speech audio at inference time significantly reduces these false positives, particularly for ambiguous or controversial statements. Crucially, the authors find that this bias stems from the classifier itself rather than transcription errors, highlighting the indispensable value of multimodal context.

What Problem Does This Paper Address?

Text toxicity models are notoriously biased against minoritized groups. Because online toxicity frequently targets marginalized communities, machine learning models often conflate the mere mention of identity markers (like "gay", "lesbian", or "Black") with toxic content, leading to high false-positive rates (FPR).

While speech-aware toxicity detection is an emerging area of interest, it hasn't been clear whether operating directly on audio mitigates or worsens these biases. Traditional cascaded systems transcribe speech to text via Automatic Speech Recognition (ASR) before running a text classifier, throwing away rich prosodic information (tone, inflection, pacing). This paper investigates whether direct access to speech data can help classifiers differentiate between actual toxicity and neutral group mentions, and isolates where the bias truly creeps into the pipeline.

Methods

The authors systematically compare four types of toxicity detection systems:

  1. ETOX: A text-only, wordlist-based baseline.
  2. DETOXIFY: A text-only neural network trained on Wikipedia and Civil Comments.
  3. MUTOX-ASR: A multimodal model trained jointly on speech and text, but restricted to using only text embeddings at inference time.
  4. MUTOX: The full multimodal model, which has access to both raw speech audio and text transcripts during inference.

To evaluate these models fairly, the authors developed a rigorous, multi-stage human annotation pipeline. Three native-level proficient annotators per language labeled samples for toxicity, specific demographic group mentions (gender, sexuality, race, religion, disability, social class), and transcript correctness. Furthermore, they re-evaluated the text-based classifiers using annotator-corrected transcripts to isolate the impact of ASR errors from classifier bias.

Datasets

The study utilizes MUTOX, a large-scale multilingual speech toxicity dataset. The authors focused on the English and Spanish test partitions, creating a new benchmark of 1,954 high-quality group annotations. This represents the first fairness audit dataset specifically designed for evaluating demographic bias in speech toxicity detection.

Key Insights and Ideas

  • Acoustic Context is a Shield Against Bias: Raw audio provides crucial prosodic cues that help models realize a group mention is benign. Models with access to audio at inference time show a marked reduction in false positives.
  • Inference-Time Speech is Mandatory: Simply training a model on speech data is not enough. If a multimodal model (MUTOX-ASR) is deprived of audio at inference time, its bias actually worsens. Without acoustic context, the model defaults to using identity terms as shortcuts for toxicity.
  • Blame the Classifier, Not the ASR: Intuitively, one might assume that bad transcriptions cause false positives. However, feeding perfect, human-corrected transcripts to text classifiers barely changes the false positive rate for group mentions. The bias is baked into the text classification logic, not the transcription pipeline.

Key Results

  • Performance Stability: Text-only models see a drop in $F_1$-score when identity groups are mentioned. In contrast, MUTOX (with speech at inference) maintains or improves precision and recall when groups are present.
F-score, precision, and recall across classifiers
  • Taming Ambiguity: For ambiguous samples (where human annotators struggled to reach a consensus), MUTOX achieved a 0% False Positive Rate (FPR). Conversely, MUTOX-ASR showed a massive spike in FPR when groups were mentioned in ambiguous contexts.
False positive rates on all vs. ambiguous samples
  • Demographic Breakdown: When looking at specific identities, MUTOX consistently reduced the FPR for mentions of women, Black, and Muslim groups compared to its speech-deprived counterpart, MUTOX-ASR.
False positive rates for specific demographic groups
  • The ASR Correction Experiment: Correcting ASR transcripts marginally improved overall $F_1$-scores for text-based models but had virtually no impact on reducing the FPR for group mentions.
Impact of transcript correction on F1 and FPR

Limitations and Future Work

  • Clip Truncation: The MUTOX dataset features short audio clips (2 to 8 seconds). This truncation often removes vital conversational context, leading to an inflated number of "Cannot say" annotations from humans.
  • Topic Skew: The authors noted a heavy skew toward liturgical/biblical content in the MUTOX dataset, which complicated gender and toxicity annotations (e.g., quoting violent or historical religious texts).
  • Language Scope: The study only covers English and Spanish (high-resource languages). The authors note that for lower-resourced languages—where ASR error rates are significantly higher—transcription errors might play a much larger role in driving pipeline bias than observed here.
  • Dialectal Nuances: Annotators were discouraged from inferring speaker demographics, making it difficult to properly evaluate reappropriated language (such as African American English).

Summary updated

Abstract

Text toxicity detection systems exhibit significant biases, producing disproportionate rates of false positives on samples mentioning demographic groups. But what about toxicity detection in speech? To investigate the extent to which text-based biases are mitigated by speech-based systems, we produce a set of high-quality group annotations for the multilingual MuTox dataset, and then leverage these annotations to systematically compare speech- and text-based toxicity classifiers. Our findings indicate that access to speech data during inference supports reduced bias against group mentions, particularly for ambiguous and disagreement-inducing samples. Our results also suggest that improving classifiers, rather than transcription pipelines, is more helpful for reducing group bias. We publicly release our annotations and provide recommendations for future toxicity dataset construction.
BibTeX
@inproceedings{bell2025role,
  author = {Samuel J. Bell and Mariano Coria Meglioli and Megan Richards and Eduardo Sánchez and Christophe Ropers and Skyler Wang and Adina Williams and Levent Sagun and Marta R. Costa-jussà},
  title = {{On the Role of Speech Data in Reducing Toxicity Detection Bias}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-long.67},
}
Figure