Political Homophily in Independence Movements: Analysing and Classifying Social Media Users by National Identity

Arkaitz Zubiaga, Bo Wang, Maria Liakata, Rob ProcterarXiv 2017 · arXiv

TL;DR

This paper introduces a language-independent approach to classify social media users as either supporting or opposing independence movements in Catalonia, the Basque Country, and Scotland. By leveraging users' follow networks rather than the text of their tweets, the authors demonstrate that political homophily is deeply embedded in digital structures, enabling stance classification with up to 97% accuracy.

What Problem Does This Paper Address?

While predicting election outcomes or general political partisanship (like Democrat vs. Republican) on social media is well-studied, analyzing national identity in regions with active independence movements presents a unique challenge. Unlike standard political affiliations, national identities are deeply tied to cultural, linguistic, and historical backgrounds, and do not always align with traditional party lines.

Furthermore, social media users rarely explicitly state their socio-demographic or political identities in a structured format. The authors tackle this gap by developing a semi-automated methodology to identify and classify pro- and anti-independence users across three distinct territories, exploring whether political homophily—the tendency to associate with like-minded individuals—manifests strongly enough in user networks to allow accurate classification.

Methods

The authors developed a multi-step framework spanning data collection, feature engineering, and supervised classification:

  1. Ground Truth Bootstrapping: To label users without manual annotation, the authors used self-reported location strings. For instance, Catalan users claiming to live in "Països Catalans" (Catalan Countries) were labeled pro-independence, while those listing "Catalunya/Cataluña along with Espanya/España" were labeled anti-independence. For Scotland, where naming conventions don't differ cleanly, they used a combination of 2014 referendum hashtags (e.g., #YesScotland vs. #BetterTogether) and location filters.
  2. Feature Extraction: They extracted four types of features to compare their predictive power:
  3. Timeline: Word2Vec embeddings representing the text of a user's recent tweets.
  4. Favourites: Word2Vec embeddings of the tweets a user liked.
  5. Interactions: A count vector of a user's retweets and replies.
  6. Network: A binary vector indicating whether a user follows or is followed by the top 1% most frequent users in the dataset.
  7. Classification: They trained and evaluated four standard classifiers—Naive Bayes, Support Vector Machines (SVM), Random Forests, and Maximum Entropy (ME)—using a stratified 10-fold cross-validation approach.

Datasets

The study yielded a substantial dataset of over 36,000 Twitter users across three territories, collected between March 2015 and February 2016:

  • Catalonia: 10,960 users (2,361 pro-independence, 8,599 anti-independence)
  • Basque Country: 7,410 users (5,377 pro-independence, 2,033 anti-independence)
  • Scotland: 18,239 users (13,114 pro-independence, 5,125 anti-independence)

The final collection comprised 27.4 million tweets (including timelines and favorites) and 19.1 million unique users within the follow networks.

Key Insights and Ideas

  • The Power of the Follow Network: Who you follow is a much stronger indicator of your national identity than what you actually say. Network-based features consistently outperformed text-based features (timelines and favorites).
  • Visualizing Homophily: As shown in the paper's network visualizations, users are heavily clustered with others who share their national identity. This separation is particularly stark in Catalonia and the Basque Country, whereas Scottish users tend to interact across boundaries slightly more often.
Fig. 1: Interactions and network connections within and across national identities. Blue: pro-independence; Red: anti- independence.
  • Subtle Behavioral Cues: The authors found fascinating digital proxies for national identity. Pro-independence users significantly favor regional top-level domains (TLDs) like .cat, .eus, and .scot in their profile URLs, while anti-independence users stick to official state TLDs like .es and .uk.
  • Echo Chambers are Friendlier: A sentiment analysis of user interactions revealed that users tweet positively 67.7% more often when interacting with members of their own group than when interacting across groups.

Key Results

The Maximum Entropy classifier leveraging Network features achieved the highest accuracy across all three regions, proving that a language-independent network approach is highly effective:

  • Catalonia: 97.2% accuracy
  • Basque Country: 90.3% accuracy
  • Scotland: 84.9% accuracy

Text-based classification (Timeline) performed reasonably well in Catalonia (94.4%) but dropped significantly in the Basque Country (83.1%) and Scotland (72.4%), underscoring the limitations of content-based analysis in multilingual or nuanced political landscapes.

Limitations and Future Work

While highly accurate, the data collection method relies on users providing specific, identifiable location strings or hashtags to establish the initial ground truth. Users with ambiguous or empty location fields cannot be easily captured by this bootstrapping technique.

Additionally, the linguistic analysis for Scotland was constrained because Twitter does not support account configurations in Scottish Gaelic or Scots, forcing the authors to rely on standard English language variants (en-gb vs en).

For future work, the authors plan to test and extend this annotation and collection approach to other globally contested territories, such as Palestine or Kurdistan.

Summary updated

Abstract

Social media and data mining are increasingly being used to analyse political and societal issues. Here we undertake the classification of social media users as supporting or opposing ongoing independence movements in their territories. Independence movements occur in territories whose citizens have conflicting national identities; users with opposing national identities will then support or oppose the sense of being part of an independent nation that differs from the officially recognised country. We describe a methodology that relies on users' self-reported location to build large-scale datasets for three territories -- Catalonia, the Basque Country and Scotland. An analysis of these datasets shows that homophily plays an important role in determining who people connect with, as users predominantly choose to follow and interact with others from the same national identity. We show that a classifier relying on users' follow networks can achieve accurate, language-independent classification performances ranging from 85% to 97% for the three territories.
BibTeX
@misc{zubiaga2017political,
  author = {Arkaitz Zubiaga and Bo Wang and Maria Liakata and Rob Procter},
  title = {{Political Homophily in Independence Movements: Analysing and Classifying Social Media Users by National Identity}},
  year = {2017},
  eprint = {1702.08388},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1702.08388},
}
Figure