TL;DR
This paper tackles the challenge of detecting offensive language in Chinese, which is often obscured by homophonic substitutions. They introduce a new dataset, HED-COLD, and a novel "homophone-aware pretraining strategy" that significantly improves the robustness of large language models against these phonetic attacks, achieving state-of-the-art results.
What Problem Does This Paper Address?
Online content moderation is crucial for maintaining healthy digital spaces, and detecting offensive language is a key part of this. While large language models (LLMs) have made great strides in this area, they struggle with languages like Chinese where users intentionally use homophones (words that sound alike but have different meanings and spellings) to evade detection. This "phonetic obfuscation" creates a blind spot for existing models, leading to a significant drop in performance when faced with these disguised offensive expressions. The authors aim to bridge this gap by developing methods that make LLMs more resilient to these homophonic attacks.
Methods
The core of their approach involves two main components:
- HED-COLD Dataset Construction: They created a new dataset, HED-COLD, by taking existing Chinese offensive language data (from the COLD dataset) and systematically applying homophonic substitutions. This process involved:
- Homophone Dictionary: Building a dictionary of homophones based on common Chinese input methods (like Google Input Method and Wubi).
- Lexical Replacement: Replacing words in original sentences with their homophonic equivalents.
- Syntactic Rewriting: Slightly altering sentence structures to create more diverse variations.
- Semantic Filtering: Crucially, they used Sentence-BERT to ensure that the meaning of the original sentence was preserved after the substitutions, filtering out pairs that became semantically inconsistent.
- Homophone-Aware Pretraining Strategy: They developed a novel pretraining strategy that leverages both the original COLD dataset and their new HED-COLD dataset. This strategy aims to align the semantic understanding of original and homophone-perturbed sentences:
- Input Mixing: Combining data from both datasets for training.
- Semantic Alignment: Using supervised fine-tuning (SFT) to train the model to produce the same offensiveness judgment and semantic interpretation for both original and homophone-substituted sentences.
- Binary Classification Output: Simplifying the output to a binary (offensive/non-offensive) prediction for efficient real-time moderation.
Datasets
- COLD (Chinese Offensive Language Dataset): Used as the base dataset for constructing HED-COLD and for fine-tuning models.
- HED-COLD (Homophone-Enhanced Dataset for Chinese Offensive Language Detection): A newly introduced dataset specifically designed to address homophonic attacks. It contains 10,000 sentences, with 7,000 for training/development and 3,000 for testing.
- ToxiCloakCN: Used as an external benchmark to evaluate the generalization capabilities of their approach on a different adversarial toxicity detection dataset.
Key Insights and Ideas
- Homophonic attacks are a significant blind spot: The paper clearly demonstrates that standard offensive language detection models perform poorly when faced with homophone-based obfuscation, highlighting the need for specialized solutions.
- Semantic preservation is key: Simply replacing words with homophones isn't enough; ensuring the original meaning is retained is critical for creating effective training data and for the model to learn meaningful associations.
- A multi-level mapping is required: To combat homophonic attacks, models need to go beyond simple lexical matching and develop a "tri-level mapping" that connects phonetic form, orthographic structure, and semantic meaning.
- Targeted data augmentation is highly effective: The HED-COLD dataset and the homophone-aware pretraining strategy show that strategically augmenting data with phonetic variations can lead to substantial improvements, especially in sensitive categories like gender and region.
Key Results
- The proposed homophone-aware pretraining strategy achieved state-of-the-art performance on both the COLD test set and the ToxiCloakCN benchmark.
- Models fine-tuned with their approach showed significant improvements in precision, recall, and F1-score, particularly on the HED-COLD test set. For example, one model's F1-score on HED-COLD improved by over 34% compared to its performance without the homophone enhancement.
- The method demonstrated balanced robustness improvements across sensitive categories like gender, region, and race, even improving performance in categories where baseline models showed degradation.
- On the ToxiCloakCN benchmark, models trained with HED-COLD showed substantial performance gains, especially on the "Homophone" subset, underscoring the effectiveness of the augmentation in enhancing attack resilience.
Limitations and Future Work
- Reliance on curated dictionaries: The current approach depends on predefined pinyin similarity rules and curated dictionaries, which might not capture the full complexity of real-world phonetic variations (e.g., ambiguous pronunciations, informal expressions).
- Task specificity: The current work focuses exclusively on offensive language detection. Its applicability to other NLP tasks like sentiment analysis, rumor detection, or dialogue moderation remains to be explored.
- Annotation guidelines: The model's performance is tied to the specific annotation guidelines of the datasets used, which can vary across platforms and cultures.
- Future Work: The authors plan to investigate multimodal homophone attacks (combining phonetic changes with visual/structural noise) and develop adaptive adversarial training pipelines that integrate phonological knowledge for more robust defense systems.
Summary updated