Beyond Binary Toxicity and Hate Speech Classification

32 concepts27 papers1 proposal

Early automated content moderation systems treated toxic language as a simple binary problem: a piece of text was either flagged or it wasn't. While computationally convenient, this "one-size-fits-all" approach creates a severe bottleneck for machine learning models. Collapsing diverse phenomena like structural racism, political incivility, and vulgarity into a single monolithic label discards the rich semantic variance necessary for models to truly understand human discourse.

As Hong et al. observe, forcing distinct categories of harm into a binary mold causes a significant loss of conceptual granularity, which ultimately degrades a model's ability to generalize across different platforms and domains. To build moderation tools that are robust, legally viable, and socially aware, the field is shifting toward fine-grained, multi-dimensional, and continuous representations of online harm.

Disentangling Targets and Intersectional Identities

To understand why a statement is harmful, a model must first resolve who is being targeted and what attributes are being weaponized against them. A binary classifier can flag an offensive phrase, but it cannot identify the demographic group under attack.

Shvets et al. argue that effective counter-strategies require a granular breakdown that separates the specific entities being attacked (the targets) from the particular characteristics being ascribed to them (the aspects). Without this distinction, moderation systems cannot provide targeted interventions or understand the specific nature of the rhetoric.

Furthermore, real-world harassment rarely targets demographic groups in isolation. Traditional annotation schemes often fail to capture intersectional targets—such as distinguishing between an attack directed at "Black women" as a distinct intersectional identity versus separate attacks on "Black people" and "women" (Sachdeva et al.).

By building datasets that explicitly map these fine-grained targets, such as the AraTar corpus introduced by Alghamdi et al. or the BD-SHS dataset by Romim et al., researchers allow models to recognize which specific communities are being disproportionately targeted and uncover the underlying social biases driving the abuse.

The Continuum of Intensity and Rhetoric

Harmful language does not exist as a discrete, binary switch; it operates on a continuous spectrum. A comment can range from a mildly offensive colloquialism to a severe, legally actionable threat.

[Mildly Offensive] ---------> [Obscene/Vulgar] ---------> [Severely Threatening]
       ^                              ^                            ^
       |                              |                            |
  Forcing this continuous spectrum into a 0-or-1 binary label destroys nuance.

When models are forced to evaluate language through a binary lens, they struggle to differentiate between these levels of severity. Bohra et al. emphasize that this flat classification fails to capture the nuanced spectrum of real-world social media text. To address this, Ayele et al. demonstrate that treating hate speech as a continuous variable rather than a discrete category yields a much more accurate reflection of online discourse.

This shift from discrete categories to continuous scaling allows models to capture subtle variations in language. For example, Eder et al. utilize a best-worst scaling approach to map the precise degrees of obscenity and vulgarity in text, moving away from rigid categorical boundaries.

This need to capture functional variation extends beyond toxicity detection into broader social language modeling. As Abulimiti et al. note, binary classification schemes fail to distinguish between the diverse functional roles and subcategories of linguistic strategies like hedging, proving that granularity is essential for capturing any complex human interaction.

Modeling Discursive Aspects and Social Logic

To move beyond superficial keyword matching, models need to reason about the underlying social logic and communicative mechanisms of a statement. A phrase can be deeply harmful without containing a single explicit slur.

To capture these implicit dynamics, researchers are developing multi-layered and hierarchical frameworks:

  • Discursive Factoring: Ron et al. introduce frameworks that break hate speech down into distinct discursive aspects, exposing the specific mechanisms through which hate is communicated rather than treating it as a monolithic expression.
  • Taxonomies of Incivility: Instead of relying on broad toxicity metrics, Gao et al. model online incivility by categorizing it into precise behaviors like aspersion, name-calling, and vulgarity, revealing how different rhetorical strategies uniquely impact online communities.
  • Hierarchical Labeling: By implementing hierarchical classification schemes, the authors of A Hierarchically-Labeled Portuguese Hate Speech Dataset and Ibrohim et al. allow models to capture the intersecting nature of harmful discourse, mapping the type, target, and severity of an utterance simultaneously.

This structural approach provides the scaffolding models need to reason about what language implies. For instance, the Social Bias Frames framework moves away from binary labels by providing a structured format to model the implied meanings and power dynamics embedded in language, forcing the model to process the underlying social logic of why a statement contains bias.

The push for semantic granularity is not just a theoretical exercise; it is an operational necessity for content moderation and legal compliance.

In legal contexts, broad terms like "toxicity" are virtually useless. Kums et al. point out that standard benchmarks lack the precise granularity required for legal assessments, which demand highly nuanced distinctions to determine actual criminal relevance. Similarly, during highly polarized global events or within multimodal contexts, binary models fail to separate broad political propaganda from targeted, dangerous harassment (Thapa et al., Jigar et al.).

Finally, embracing granularity fundamentally changes how we collect data. By drawing on fields like Critical Discourse Analysis, Assimakopoulos et al. demonstrate that multi-layered annotation schemes actually improve inter-annotator agreement. When human labelers are given a structured, nuanced framework rather than a reductive binary choice, they can more reliably identify and categorize the complex, implicit, and non-explicit forms of discriminatory language that dominate the internet today.

Key Foundational Work

Multi-label Hate Speech and Abusive Language Detection in Indonesian Twitter (2019) — contributed 2 concept elements to this direction.
A Hierarchically-Labeled Portuguese Hate Speech Dataset (2019) — contributed 1 concept elements to this direction.
A Dataset of Hindi-English Code-Mixed Social Media Text for Hate Speech Detection (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_binary_classification_limitations_6f3fab26,
  title = {binary classification limitations},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/binary-classification-limitations/rdr_6f3fab26d30641b2b627971c0ff35f9b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_binary_classification_limitations_6f3fab26_v1,
  title = {binary classification limitations},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/binary-classification-limitations/rdr_6f3fab26d30641b2b627971c0ff35f9b/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake