For Women, Life, Freedom: A Participatory AI-Based Social Web Analysis of a Watershed Moment in Iran's Gender Struggles

Adel Khorramrouz, Sujan Dutta, Ashiqur R. KhudaBukhshIJCAI 2023 · DOI

TL;DR

This paper presents a computational analysis of Persian Twitter discourse surrounding the "Woman, Life, Freedom" protests in Iran, triggered by Mahsa Amini's death. It introduces a novel participatory AI approach, involving Iranian women as active annotators, to understand shifts in gender equality sentiment and identify key themes.

What Problem Does This Paper Address?

The paper tackles the challenge of understanding nuanced social and political movements through social media, specifically focusing on the "Woman, Life, Freedom" protests in Iran. Existing computational analyses often lack cultural context and stakeholder involvement, potentially leading to biased interpretations. The authors aim to bridge this gap by analyzing Persian Twitter discourse to gauge the impact of Mahsa Amini's death on gender equality sentiment, while crucially incorporating the lived experiences of Iranian women into the AI model development process. This approach is particularly important given the sensitive nature of the topic and the potential for state-aligned narratives to dominate online discussions.

Methods

The core of the paper's methodology lies in its participatory AI approach and an ensemble active learning pipeline.

  • Participatory AI: Unlike traditional annotation where annotators merely provide labels, this project actively involves Iranian women as annotators. They not only label tweets but also contribute by suggesting relevant keywords for data curation and providing short example documents to guide the active learning process. This ensures the AI system is built with cultural context and first-hand experience.
  • Ensemble Active Learning Pipeline: This pipeline is designed to efficiently build a robust classifier for detecting stance towards gender equality. It involves several stages:
  • Seed Set Construction: This begins with random sampling from general Persian Twitter data and a specifically curated "gender" dataset. Crucially, it also employs guided sampling, where annotators provide representative short documents for positive and negative stances, creating a richer and more culturally grounded initial dataset.
  • Certainty Sampling: This step focuses on identifying instances where the model is highly confident about its predictions (both positive and negative) to refine its understanding of clear-cut cases.
  • Uncertainty Sampling (Margin Sampling): This strategy selects instances where the model is least confident, pushing the boundaries of its knowledge and improving its performance on ambiguous cases.

Datasets

The study utilizes several custom-created datasets:

  • Dgender: A dataset of over 6 million tweets related to gender, including keywords like "woman," "girl," "namoos" (honor/female family members), and "gheirat" (male protectiveness/jealousy), as well as gendered insults.
  • Dprotest: A dataset of over 21 million tweets containing the #MahsaAmini hashtag, along with tweets supporting the government (#ISupportKhamenei) and calling for executions (#ExecuteThem).
  • Dbaseline: A dataset of approximately 2.2 million tweets used to represent typical Persian Twitter activity before and after the protest period, serving as a benchmark.

The paper also describes the creation of a seed set for the active learning pipeline, which was expanded through guided and random sampling, resulting in a dataset with 343 positive, 440 negative, and 1,051 neutral instances.

Key Insights and Ideas

  • The Power of Participatory AI: The paper strongly advocates for involving domain experts and affected communities (in this case, Iranian women) in the AI development lifecycle. Their insights are crucial for creating culturally relevant and accurate models, especially for sensitive topics.
  • Polarization and Increased Discourse: The Mahsa Amini protests led to a significant increase in both positive and negative discourse surrounding gender equality on Persian Twitter. This suggests a heightened engagement with the topic, rather than a simple shift in one direction.
  • Pro-Protest Accounts Mirror Baseline Activity: Interestingly, accounts associated with the protests showed account creation time distributions more similar to general Persian Twitter users, suggesting they weren't solely created for the protest, unlike potentially state-aligned accounts.
  • Cultural Nuances in Keywords: The annotators' suggestions for keywords like "namoos" and "gheirat" highlight the deep-seated cultural concepts that underpin discussions about gender roles and equality in Iran, which might be missed by standard keyword selection.
  • Twitter as a Double-Edged Sword: While Twitter served as a vital platform for protest and expression, the paper notes the increasing challenges of accessing and analyzing this data due to platform changes and potential censorship, posing a significant hurdle for future research.

Key Results

  • The participatory approach yielded a richer seed set for active learning compared to a random baseline.
  • The analysis revealed a noticeable shift towards increased polarization in Persian Twitter discourse regarding gender equality after Mahsa Amini's death.
  • The increase in positive tweets related to gender equality was slightly greater than the increase in negative tweets during the protest period.
  • The trained classifier achieved a Macro F1 performance of 73.27%, with a binary classifier (positive vs. not-positive) reaching 77.76%.
  • Account creation time distributions showed that pro-protest accounts were more similar to baseline Persian Twitter activity than state-aligned or pro-execution accounts.

Limitations and Future Work

  • Data Accessibility: The paper acknowledges the growing uncertainty around the future accessibility of Twitter data due to platform policy changes, which could impact the reproducibility and scope of future analyses.
  • Annotation Subjectivity: While inter-rater agreement was measured and disagreements were resolved, the subjective nature of stance detection remains a challenge.
  • Bias in Large Language Models: The authors note that their model was trained on a large language model, which can inherit biases from its training data, a concern that warrants further investigation.

Future work could involve exploring more sophisticated methods for handling data access challenges, further investigating the biases within LLMs used for social analysis, and expanding the participatory approach to other social media platforms and contexts.

Summary updated

BibTeX
@inproceedings{khorramrouz2023women,
  author = {Adel Khorramrouz and Sujan Dutta and Ashiqur R. KhudaBukhsh},
  title = {{For Women, Life, Freedom: A Participatory AI-Based Social Web Analysis of a Watershed Moment in Iran's Gender Struggles}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/667},
}
Figure