Traditional text classification treats documents or sentences as monolithic blocks. For tasks like content moderation, propaganda detection, or identifying social media manipulation, a binary label (e.g., "toxic" or "manipulative") states that a text is problematic, but fails to explain where or why. To build interpretable and actionable systems, researchers have shifted toward fine-grained span detection—identifying the exact character or token offsets responsible for a text's classification.
This paradigm is central to benchmarks like the SemEval-2021 Toxic Spans Detection task (SemEval-2021 Task 5: Toxic Spans Detection) organized by John Pavlopoulos et al., as well as tasks targeting propaganda technique detection (Nullpointer at ArAIEval Shared Task: Arabic Propagandist Technique Detection with Token-to-Word Mapping in Sequence Tagging) and social media manipulation (The UNLP 2025 Shared Task on Detecting Social Media Manipulation).
Framing Span Detection as Sequence Labeling
The core intuition behind solving this problem lies in transforming a span extraction task into a sequence labeling problem, directly analogous to Named Entity Recognition (NER). Instead of predicting a single label for an entire sequence, the problem is broken down into a series of token-level classification decisions.
As Harsh Kataria et al. demonstrate in BennettNLP at SemEval-2021 Task 5: Toxic Spans Detection using Stacked Embedding Powered Toxic Entity Recognizer, treating toxic or offensive phrases as specific "entities" allows models to leverage established sequence tagging architectures. Tokens can be classified using a simple binary schema (e.g., toxic vs. non-toxic), an approach utilized by Abdessamad Benlahbib et al. in LISAC FSDM USMBA at SemEval-2021 Task 5: Tackling Toxic Spans Detection Challenge with Supervised SpanBERT-based Model and Unsupervised LIME-based Model.
Alternatively, to capture precise boundaries, tokens can be tagged using positional schemas like BISO (Begin, Inner, Single, Out). In hub at SemEval-2021 Task 5: Toxic Span Detection Based on Word-Level Classification, the authors show that word-level classification using these explicit boundary roles enables highly granular localization of harmful segments.
Contextual Embeddings and Sequence Dependencies
Token classification succeeds because modern Pre-trained Language Models (PLMs) generate contextualized representations. The embedding for a specific token changes depending on its neighbors, allowing the model to recognize when an otherwise benign word is being used maliciously.
Typically, a PLM like BERT, RoBERTa, or ALBERT is fine-tuned with a token classification objective, where a linear layer maps each token's final hidden state to the target label space (YoungSheldon at SemEval-2021 Task 5: Fine-tuning Pre-trained Language Models for Toxic Spans Detection using Token classification Objective; SRPOL DIALOGUE SYSTEMS at SemEval-2021 Task 5: Automatic Generation of Training Data for Toxic Spans Detection). To capture broader sequence-level constraints and prevent illegal label transitions (such as an "Inner" token appearing without a preceding "Begin" token), researchers frequently append a Conditional Random Field (CRF) or a BiLSTM-CRF layer (hub at SemEval-2021 Task 5: Toxic Span Detection Based on Word-Level Classification; CSECU-DSG at SemEval-2021 Task 5: Leveraging Ensemble of Sequence Tagging Models for Toxic Spans Detection).
Furthermore, performance can be amplified by combining multiple embedding spaces. For instance, the system designed by Harsh Kataria et al. stacks transformer-based embeddings (BERT) with character-level contextual embeddings (Flair) and static embeddings (GloVe). This hybrid representation captures both deep semantic context and surface-level morphological regularities, making the token classifier highly robust to spelling variations and social media noise.
Overcoming Fragmentation with Continuity Heuristics
A major challenge of token-level classification is span fragmentation. Because models evaluate tokens or sub-tokens semi-independently, a continuous multi-word toxic phrase might be predicted as a series of disjointed toxic tokens separated by "non-toxic" whitespace or punctuation.
To enforce structural coherence, researchers employ post-processing heuristics that bridge these artificial gaps. In Lone Pine at SemEval-2021 Task 5: Fine-Grained Detection of Hate Speech Using BERToxic, Yakoob Khan et al. introduce a gap-filling heuristic. If consecutive tokens are predicted as toxic, the system automatically labels the intervening characters (such as spaces or punctuation marks) as toxic. This ensures that fragmented token predictions are consolidated into unified, continuous character spans that align with human annotation.
Similarly, in Nullpointer at ArAIEval Shared Task: Arabic Propagandist Technique Detection with Token-to-Word Mapping in Sequence Tagging, the authors handle sub-word tokenization by mapping predictions back to word boundaries, specifically focusing on the first token of each word to maintain structural alignment.
Robustness Across Languages and Code-Mixed Text
The sequence labeling framework has proven remarkably adaptable to low-resource and linguistically complex scenarios, such as code-mixed social media text. In tasks like offensive span identification for Tamil-English (Findings of the Shared Task on Offensive Span Identification fromCode-Mixed Tamil-English Comments) and Kannada-English (Findings of the First Shared Task on Offensive Span Identification from Code-Mixed Kannada-English Comments), the authors frame the problem as supervised sequence labeling to pinpoint exact character offsets.
Similarly, token classification has been successfully adapted to Turkish hate speech detection (HATECAT-TR: A Hate Speech Span Detection and Categorization Dataset for Turkish). Because the classification happens at the token level, models can isolate specific offensive loanwords or code-switched expressions without requiring a complete syntactic parse of the mixed languages, demonstrating the universal utility of the sequence labeling paradigm.
Key Foundational Work
YoungSheldon at SemEval-2021 Task 5: Fine-tuning Pre-trained Language Models for Toxic Spans Detection using Token classification Objective (2021) — contributed 2 concept elements to this direction.
MIPT-NSU-UTMN at SemEval-2021 Task 5: Ensembling Learning with Pre-trained Language Models for Toxic Spans Detection (2021) — contributed 2 concept elements to this direction.
BennettNLP at SemEval-2021 Task 5: Toxic Spans Detection using Stacked Embedding Powered Toxic Entity Recognizer (2021) — contributed 2 concept elements to this direction.