Abstract
Building on previous research, this study assesses how social solidarity towards migrants and refugees shifted before and after the onset of the COVID-19 pandemic. We analyzed a novel longitudinal dataset of over 240,000 migration-related tweets collected between September 2019 and June 2021. By training a BERT model using expert and crowd-sourced annotations alongside data augmentation strategies, we achieved performance near the human upper bound. Our findings reveal that while migrant solidarity became increasingly salient and contested during the early stages of the pandemic, it declined by late 2020. Crucially, we demonstrate that the perceived ratio of solidarity to anti-solidarity is heavily influenced by "design choices," such as tweet language, user anonymity, and sampling methods.
Introduction
Social scientists and policymakers frequently utilize social media to gauge public sentiment during crises. However, existing research often suffers from several critical "blind spots":
- Lack of Baselines: Studies frequently begin only after a crisis starts, making it impossible to distinguish new behaviors from pre-existing trends.
- One-Sided Focus: Literature often prioritizes anti-social behavior (e.g., hate speech) over the interplay between pro-social (solidarity) and anti-social expressions.
- Short Timeframes: Brief observation windows miss the long-term evolution of social dynamics.
- Sampling Bias: Choices regarding language or hashtags can inadvertently bake bias into conclusions.
This paper addresses these gaps by providing a rare "pre-crisis" baseline and analyzing the long-term development of migrant-related discourse through the COVID-19 era.
Methodology
We employed a multi-stage pipeline to bridge the gap between small-scale human annotation and large-scale automated analysis. Initially, ~2,000 tweets were categorized by experts and crowd workers into solidarity, anti-solidarity, ambivalent, or not applicable.
To scale this, we utilized multilingual BERT (mBERT) and XLM-R for 3-way classification. To reach human-level performance, we implemented sophisticated data augmentation strategies, including back-translation (creating synthetic training variations) and self-learning. In the latter, an ensemble of models labeled 105,000 additional tweets; those with high agreement served as a "silver standard" to retrain the final model, boosting the Macro-F1 score from ~60% to ~80%.
Dataset Characteristics
The final extended dataset comprises 243,812 tweets. As shown in the table below, the majority of the discourse was in German (61.2%) and focused specifically on refugees (94.5%).
Temporal Trends and Crisis Effects
Our analysis shows that migrant solidarity spiked during the early stages of the pandemic (March 2020) but declined steadily thereafter. By summer 2021, tweet volumes had fallen slightly below pre-pandemic levels. Interestingly, peaks in solidarity were often tied to specific humanitarian events—such as the fire at the Moria refugee camp in September 2020—rather than the pandemic itself.
While the COVID-19 pandemic "infected" the migration discourse in March 2020 (with over 40% of migration tweets mentioning the virus), this association was short-lived, returning to low levels by summer 2020 even as the health crisis persisted.
The Role of Design Choices
A central contribution of this work is identifying how sampling design alters findings:
- The "Solidarity Bias" of Hashtags: Tweets without hashtags are significantly more likely to express anti-solidarity. Sampling only by popular hashtags (e.g., #RefugeesWelcome) leads to an over-reporting of social solidarity.
- Anonymity and Hostility: "Anonymous" users (those without location data) writing in German were more likely to post anti-solidarity content than identified users.
- Language as Identity: In Germany, English-language tweets exhibited much higher levels of solidarity than native German tweets, suggesting that language choice is intertwined with national identity and education levels.
Conclusion
These findings highlight the necessity of long-term observation and careful sampling in crisis research. While social media offers a window into public sentiment, researchers must account for the biases inherent in their design choices. Future work should integrate representative survey data to further calibrate social media findings and explore dynamics across different platforms.
Summary updated