TL;DR
This paper introduces a language-independent approach to classify social media users as either supporting or opposing independence movements in Catalonia, the Basque Country, and Scotland. By leveraging users' follow networks rather than the text of their tweets, the authors demonstrate that political homophily is deeply embedded in digital structures, enabling stance classification with up to 97% accuracy.
What Problem Does This Paper Address?
While predicting election outcomes or general political partisanship (like Democrat vs. Republican) on social media is well-studied, analyzing national identity in regions with active independence movements presents a unique challenge. Unlike standard political affiliations, national identities are deeply tied to cultural, linguistic, and historical backgrounds, and do not always align with traditional party lines.
Furthermore, social media users rarely explicitly state their socio-demographic or political identities in a structured format. The authors tackle this gap by developing a semi-automated methodology to identify and classify pro- and anti-independence users across three distinct territories, exploring whether political homophily—the tendency to associate with like-minded individuals—manifests strongly enough in user networks to allow accurate classification.
Methods
The authors developed a multi-step framework spanning data collection, feature engineering, and supervised classification:
- Ground Truth Bootstrapping: To label users without manual annotation, the authors used self-reported location strings. For instance, Catalan users claiming to live in "Països Catalans" (Catalan Countries) were labeled pro-independence, while those listing "Catalunya/Cataluña along with Espanya/España" were labeled anti-independence. For Scotland, where naming conventions don't differ cleanly, they used a combination of 2014 referendum hashtags (e.g.,
#YesScotlandvs.#BetterTogether) and location filters. - Feature Extraction: They extracted four types of features to compare their predictive power:
- Timeline: Word2Vec embeddings representing the text of a user's recent tweets.
- Favourites: Word2Vec embeddings of the tweets a user liked.
- Interactions: A count vector of a user's retweets and replies.
- Network: A binary vector indicating whether a user follows or is followed by the top 1% most frequent users in the dataset.
- Classification: They trained and evaluated four standard classifiers—Naive Bayes, Support Vector Machines (SVM), Random Forests, and Maximum Entropy (ME)—using a stratified 10-fold cross-validation approach.
Datasets
The study yielded a substantial dataset of over 36,000 Twitter users across three territories, collected between March 2015 and February 2016:
- Catalonia: 10,960 users (2,361 pro-independence, 8,599 anti-independence)
- Basque Country: 7,410 users (5,377 pro-independence, 2,033 anti-independence)
- Scotland: 18,239 users (13,114 pro-independence, 5,125 anti-independence)
The final collection comprised 27.4 million tweets (including timelines and favorites) and 19.1 million unique users within the follow networks.
Key Insights and Ideas
- The Power of the Follow Network: Who you follow is a much stronger indicator of your national identity than what you actually say. Network-based features consistently outperformed text-based features (timelines and favorites).
- Visualizing Homophily: As shown in the paper's network visualizations, users are heavily clustered with others who share their national identity. This separation is particularly stark in Catalonia and the Basque Country, whereas Scottish users tend to interact across boundaries slightly more often.
- Subtle Behavioral Cues: The authors found fascinating digital proxies for national identity. Pro-independence users significantly favor regional top-level domains (TLDs) like
.cat,.eus, and.scotin their profile URLs, while anti-independence users stick to official state TLDs like.esand.uk. - Echo Chambers are Friendlier: A sentiment analysis of user interactions revealed that users tweet positively 67.7% more often when interacting with members of their own group than when interacting across groups.
Key Results
The Maximum Entropy classifier leveraging Network features achieved the highest accuracy across all three regions, proving that a language-independent network approach is highly effective:
- Catalonia: 97.2% accuracy
- Basque Country: 90.3% accuracy
- Scotland: 84.9% accuracy
Text-based classification (Timeline) performed reasonably well in Catalonia (94.4%) but dropped significantly in the Basque Country (83.1%) and Scotland (72.4%), underscoring the limitations of content-based analysis in multilingual or nuanced political landscapes.
Limitations and Future Work
While highly accurate, the data collection method relies on users providing specific, identifiable location strings or hashtags to establish the initial ground truth. Users with ambiguous or empty location fields cannot be easily captured by this bootstrapping technique.
Additionally, the linguistic analysis for Scotland was constrained because Twitter does not support account configurations in Scottish Gaelic or Scots, forcing the authors to rely on standard English language variants (en-gb vs en).
For future work, the authors plan to test and extend this annotation and collection approach to other globally contested territories, such as Palestine or Kurdistan.
Summary updated