Leveraging Dependency Grammar for Fine-Grained Offensive Language Detection using Graph Convolutional Networks

Divyam Goel, Raksha SharmaNAACL 2022

TL;DR

This paper introduces SyLSTM, a novel deep learning architecture that combines semantic and syntactic information to improve offensive language detection on social media. By leveraging dependency grammar through Graph Convolutional Networks (GCNs), SyLSTM achieves state-of-the-art results while being significantly more parameter-efficient than existing models like BERT.

What Problem Does This Paper Address?

The proliferation of offensive content on social media poses a significant challenge for online platforms. While existing methods for detecting offensive language often rely on lexical features, they struggle with nuances like sarcasm and can inadvertently exhibit biases, leading to false positives and discrimination against protected groups. Furthermore, simply identifying offensive content isn't enough; understanding the type and target of the offense is crucial for effective moderation. This paper aims to address these limitations by developing a more robust and nuanced approach to offensive language detection that accounts for the grammatical structure of language.

Methods

The core of this paper is the SyLSTM architecture, which integrates two key components:

  • BiLSTM-based Semantic Encoder: This component processes word embeddings to capture semantic meaning and long-range dependencies within a tweet. It uses a Bidirectional LSTM (BiLSTM) to consider both past and future contexts, which is beneficial for detecting offensive language that can be distributed throughout a sentence.
  • GCN-based Syntactic Encoder: This is the novel aspect of the approach. It takes the dependency parse tree of a sentence and transforms it into a graph. Graph Convolutional Networks (GCNs) are then used to extract features from this graph, effectively encoding the syntactic relationships between words. To make dependency trees compatible with GCNs, the authors introduce inverse edges and self-loops to create an undirected graph representation.

These two encoders are combined, and their outputs are fed into a Feed Forward Network (FFNN) and then an output layer with a softmax head for classification. The model is designed to be parameter-efficient, aiming to outperform larger models like BERT with fewer parameters.

Datasets

The authors evaluate their SyLSTM model on two datasets:

  1. Offensive Language Identification Dataset (OLID): This dataset, originally presented for a SemEval challenge, is structured hierarchically. It annotates tweets for:
  2. Whether a tweet is offensive or not (Task A).
  3. Whether an offensive tweet is targeted (Task B).
  4. The target of the offensive tweet (individual, group, or other) (Task C). The authors specifically highlight the importance of subtasks B and C for their research objectives.

  5. Hate Speech and Offensive Language Dataset: Curated by Davidson et al. (2017), this dataset classifies tweets into three categories: hate speech (HATE), offensive language (OFF), and none (NONE). This dataset is crucial for addressing the challenge of differentiating hate speech from general offensive language.

Key Insights and Ideas

  • Syntactic structure matters: The paper strongly advocates for the inclusion of syntactic information (dependency grammar) in offensive language detection. They argue that this helps overcome biases inherent in purely lexical approaches and allows for a deeper understanding of nuanced language.
  • GCNs for dependency trees: The authors demonstrate a practical way to adapt dependency parse trees into graph structures suitable for GCNs, overcoming previous limitations in handling non-Euclidean syntactic data.
  • Complementarity of semantic and syntactic features: The SyLSTM architecture effectively combines the strengths of BiLSTMs for semantic understanding and GCNs for syntactic analysis, leading to improved performance.
  • Robustness on smaller datasets: The SyLSTM shows a significant relative improvement over BERT on the OLID dataset, suggesting its robustness even when generalizing over smaller datasets.
  • Parameter efficiency: A key takeaway is that SyLSTM achieves superior performance with orders of magnitude fewer parameters than BERT, making it a more practical solution.

Key Results

The SyLSTM model, particularly the version utilizing pre-trained GloVe-Twitter embeddings (SyLSTM*), consistently outperforms baseline models (Linear-SVM, BiLSTM, BERT) across all tasks and datasets.

  • Offensive Language Detection: SyLSTM* achieves an F1-score of 87.4%, surpassing BERT's 85.7%.
  • Categorization of Offensive Language: SyLSTM* achieves an F1-score of 93.2%, a significant 4% relative improvement over BERT.
  • Offensive Language Target Identification: SyLSTM* achieves an F1-score of 64.4%, outperforming BERT's 60.9%.
  • Hate Speech and Offensive Language Dataset: SyLSTM* achieves an F1-score of 92.7%, significantly outperforming all baselines.

The paper also notes that their model's performance is largely unaffected by minor errors in dependency parsing, highlighting the robustness of their approach.

Limitations and Future Work

  • Dependency Parser Accuracy: The authors acknowledge that existing dependency parsers may not always generate perfectly accurate trees for tweets, which can be a limitation. However, they argue that their graph transformation method mitigates this issue by focusing on congruent copies of the actual dependency tree.
  • Over-parametrization with BERT: While not a direct limitation of SyLSTM, the paper notes that combining GCNs with BERT encoders can lead to over-parametrization and sparsity issues, suggesting that SyLSTM's approach of mounting a GCN over a BERT encoder might be more efficient.
  • Future Work: The paper suggests that the SyLSTM architecture's simplicity and effectiveness make it a promising solution for incorporating graph-structured linguistic features into various NLP tasks. The authors also provide a replication package, encouraging further research and application of their method.

Summary updated

BibTeX
@inproceedings{goel2022leveraging,
  author = {Divyam Goel and Raksha Sharma},
  title = {{Leveraging Dependency Grammar for Fine-Grained Offensive Language Detection using Graph Convolutional Networks}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure