Cross-lingual Transfer Learning for COVID-19 Outbreak Alignment

Sharon Levy, William Yang Wang

TL;DR

This paper explores using Twitter data from countries with early COVID-19 outbreaks to predict outbreaks in other countries, even with different languages. They demonstrate that cross-lingual transfer learning, leveraging advanced language models and tweet features, can achieve significant correlations in predicting epidemiological trends across borders.

What Problem Does This Paper Address?

The COVID-19 pandemic highlighted the interconnectedness of global health and the challenge of predicting and managing outbreaks across diverse regions. While social media offers a rich source of real-time, unfiltered data, many existing disease detection models fail to leverage this information effectively or account for cross-country patterns. Specifically, this paper addresses the question of whether knowledge gained from analyzing social media data in one country (with an earlier outbreak) can be transferred to help understand and predict outbreaks in another country, even when they speak different languages. This is crucial for developing more robust and globally applicable epidemiological surveillance systems.

Methods

The authors propose a task of cross-lingual transfer learning for epidemiological alignment. Their approach involves training a model on data from a source country (Italy, due to its early outbreak) and then testing its predictive capabilities on target countries with different languages.

Key methodological components include:

  • Feature Engineering: They utilize both "macro" and "micro" text features. Macro features include daily tweet frequency. Micro features are derived from advanced multilingual sentence embeddings generated by models like multilingual BERT (mBERT) and LASER.
  • Semantic Alignment: mBERT and LASER are used to capture the semantic meaning of tweets, allowing for connections across languages. They explore average-pooling and max-pooling strategies for these embeddings.
  • Regression Model: A support vector regression model is employed to map text features to epidemiological outputs.
  • Ground Truth: The model predicts two types of outputs: total cumulative cases and daily new cases.
  • Evaluation Metric: Due to potential offsets in official country data, they use Spearman's correlation to measure the alignment between predicted and actual case counts.
  • Transfer Learning Setting: They train on Italy's early data and test on other countries, exploring various time settings for training and testing to simulate different stages of an outbreak.

Datasets

  • COVID-19 Twitter Dataset: This dataset, collected via Twitter's streaming API, comprises millions of tweets filtered for 22 COVID-19 related keywords and hashtags. The study focuses on tweets written in Italian, Indonesian, Turkish, Japanese, and Thai, collected from February 1st, 2020, to April 30th, 2020.
  • COVID-19 Dashboard by the CSSE at Johns Hopkins University: This serves as the ground truth for daily confirmed COVID-19 cases for each country.

Key Insights and Ideas

  • Social Media as a Predictor: The paper strongly suggests that social media, specifically tweet frequency and semantic content, can serve as a valuable indicator for epidemiological trends, even across linguistic barriers.
  • Cross-Lingual Transfer is Feasible: The core insight is that models trained on one language's social media data can indeed generalize to predict outbreak patterns in another language, demonstrating the universality of certain social media behaviors related to pandemics.
  • Language Models Bridge the Gap: State-of-the-art cross-lingual sentence embeddings (mBERT, LASER) are crucial for enabling this cross-lingual transfer, allowing for semantic alignment beyond simple keyword matching.
  • Macro and Micro Features Complement Each Other: Combining simple tweet frequency with rich semantic embeddings leads to stronger predictive power.
  • Linear Growth is Easier to Transfer: Transfer learning appears to work better during periods of more linear case increases rather than during volatile peak periods, suggesting that stable trends are more predictable across different contexts.

Key Results

  • The study achieved strong correlations in cross-country predictions, with Spearman correlations reaching up to 0.85 in some settings.
  • For the total case setting, predictions showed strong and monotonically increasing alignment with ground truth across all time settings when training on Italy and testing on other countries.
  • For the new daily case setting, correlations were generally weaker but still showed promise, particularly in specific time settings (II and III).
  • The upper bound correlation (setting V, training on Feb-Apr and testing on all three months) indicated that tweets across countries generally follow the same trend in relation to reported cases, suggesting that observed weaker correlations in other settings are due to limited data and specific time window challenges.

Limitations and Future Work

  • Limited Sample Sizes: The authors acknowledge that the early stage of the pandemic meant limited sample sizes, which could affect the strength of correlations.
  • Time Window Sensitivity: The performance of transfer learning varied depending on the training and testing time windows, especially for predicting new daily cases, which are more volatile.
  • Indonesia Anomaly: Indonesia showed different alignment patterns for new daily cases, likely because it was still in a phase of steady increase by April, unlike other countries that had reached or passed their peaks. This highlights the challenge of generalizing to different outbreak trajectories.

Future Work: The authors suggest that with larger sample sizes and by training on a wider variety of outbreak stages (pre, mid, and post-peak), stronger and more generalizable cross-country alignment models can be built. They envision future integration of social media data into epidemiological prediction systems to enhance outbreak detection.

Summary updated

Abstract

The spread of COVID-19 has become a significant and troubling aspect of society in 2020. With millions of cases reported across countries, new outbreaks have occurred and followed patterns of previously affected areas. Many disease detection models do not incorporate the wealth of social media data that can be utilized for modeling and predicting its spread. It is useful to ask, can we utilize this knowledge in one country to model the outbreak in another? To answer this, we propose the task of cross-lingual transfer learning for epidemiological alignment. Utilizing both macro and micro text features, we train on Italy's early COVID-19 outbreak through Twitter and transfer to several other countries. Our experiments show strong results with up to 0.85 Spearman correlation in cross-country predictions.
BibTeX
@inproceedings{levy2020cross,
  author = {Sharon Levy and William Yang Wang},
  title = {{Cross-lingual Transfer Learning for COVID-19 Outbreak Alignment}},
  booktitle = {NLP-COVID @ ACL},
  year = {2020},
  url = {https://openreview.net/forum?id=ub9_2iAo3D},
}
Figure