MarSan at SemEval-2023 Task 10: Can Adversarial Training with help of a Graph Convolutional Network Detect Explainable Sexism?

Ehsan Tavan, Maryam NajafiACL 2023

TL;DR

This paper introduces a novel framework for detecting online sexism by combining Graph Convolutional Networks (GCNs) with adversarial training. Their approach leverages both linguistic context from BERTweet and structural relationships captured by GCNs to achieve competitive results in fine-grained sexism detection, demonstrating the effectiveness of adversarial training for robustness.

What Problem Does This Paper Address?

Online platforms have become breeding grounds for hate speech and offensive language, with sexism being a particularly harmful form of abuse that disproportionately impacts women. Automatically detecting and classifying sexism is crucial for creating safer online spaces and informing research and policy. However, identifying sexism is challenging due to subtle language, sarcasm, microaggressions, and context-dependent interpretations. Existing methods often struggle with these nuances, and there's a need for models that are not only accurate but also robust against adversarial attacks and capable of explainable detection.

Methods

The proposed framework integrates three key components:

  • Encoder Module: This module uses BERTweet, a pre-trained language model for tweets, enhanced with a transformer and a BiLSTM layer. This combination aims to extract rich contextual and semantic features from the input text.
  • Graph Constructor: To capture structural relationships, the paper constructs three types of heterogeneous text graphs:
  • Semantic-based Graph: Captures semantic similarity between words using cosine similarity.
  • Syntactic-based Graph: Identifies syntactic dependencies between words.
  • Sequential-based Graph: Analyzes local word co-occurrence using Pointwise Mutual Information (PMI) and a sliding window strategy. These graphs are then processed by GCNs.

  • GCN Module: Multiple GCN layers are applied to the constructed graphs to generate node embeddings. The embeddings from the three graph types are then concatenated and processed to capture a comprehensive representation of the text's structure.

  • Classifier Module: A standard softmax classifier is used to predict the sexism label based on the combined features from the encoder and GCN modules.
  • Adversarial Training: To improve robustness and generalization, the authors employ Projected Gradient Descent (PGD) to introduce small perturbations to the input during training. This process is applied to both the BERTweet embeddings and the GCN node embeddings.

Datasets

The paper utilizes the dataset from SemEval-2023 Task 10, "Explainable Detection of Online Sexism" (EDOS). This dataset was manually annotated by human experts in English and includes three subtasks: binary sexism detection, category of sexism, and fine-grained vector of sexism.

Key Insights and Ideas

  • Synergy of Language and Structure: The core idea is that combining the contextual understanding from language models (BERTweet) with the structural relationships captured by GCNs on different graph types (semantic, syntactic, sequential) leads to a more comprehensive representation of sexist content.
  • Importance of Graph Construction: The paper highlights that the way graphs are constructed (e.g., through lemmatization and stop-word removal) significantly impacts performance, suggesting that careful preprocessing is crucial for graph-based methods.
  • Adversarial Training for Robustness: Adversarial training is shown to be effective in enhancing the model's resilience to subtle perturbations, leading to improved generalization and performance, especially in challenging tasks like sexism detection.
  • Fine-grained Detection Benefits: The approach demonstrates strong performance across all three subtasks, indicating its capability to handle both broad and nuanced classifications of sexism.

Key Results

The proposed model achieved competitive results in the SemEval-2023 Task 10. Specifically:

  • The model achieved a macro F1-score of 85.78% on the development set and 85.18% on the test set for Subtask A (binary sexism detection).
  • For Subtask B (category of sexism), the model achieved 70.82% on the development set and 66.09% on the test set.
  • For Subtask C (fine-grained vector of sexism), the model achieved 48.94% on the development set and 48.22% on the test set.
  • The paper also shows that the combination of the proposed encoder, GCN module, and adversarial training consistently outperformed individual components and simpler architectures across all subtasks. Adversarial training, in particular, led to noticeable improvements, such as a 2.17 percentage point increase in Subtask B.

Limitations and Future Work

  • Limitations: The paper acknowledges that while the model performs well, the fine-grained detection (Subtask C) still has room for improvement. The performance difference between the "no-lemma" and "no-remove-stopwords" models suggests that both preprocessing steps are important, but their individual impact is not drastically different, indicating potential for further optimization.
  • Future Work: The authors suggest that their approach of combining contextual features from the encoder module with graph-based features provides a reasonable range of results for explainable detection. Future work could explore more advanced graph construction techniques or different adversarial training strategies to further push the boundaries of fine-grained sexism detection and model robustness.

Summary updated

BibTeX
@inproceedings{tavan2023marsan,
  author = {Ehsan Tavan and Maryam Najafi},
  title = {{MarSan at SemEval-2023 Task 10: Can Adversarial Training with help of a Graph Convolutional Network Detect Explainable Sexism?}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure