Detecção de desinformação sobre Covid-19 no Twitter

Ana Alice Ximenes Mota, Wellington Franco, César Lincoln Cavalcante MattosSTIL 2021 · DOI

Abstract

The COVID-19 pandemic triggered an "infodemic" of misinformation, particularly on social media. This study presents the development of the first public Portuguese-language corpus specifically for COVID-19 misinformation on Twitter. By evaluating various machine learning models, researchers from the Federal University of Ceará demonstrated that a traditional Support Vector Machine (SVM) classifier using TF-IDF representation outperformed modern word embeddings, achieving an F1-score of 89% and a recall of 97%.

Introduction

The rapid spread of misleading health claims during the COVID-19 pandemic, such as "miracle cures" involving lemon water or ivermectin, posed significant public health risks. While misinformation detection has been extensively studied in English, there was a notable lack of specialized, publicly available datasets for the Portuguese language. This research addresses this gap by creating a labeled corpus of Portuguese tweets and establishing a performance baseline for automated detection systems.

Methodology

The researchers implemented a rigorous Natural Language Processing (NLP) pipeline to construct and evaluate the detection models.

Corpus Construction

The data collection process involved four main stages:

  1. Selection of fake topics: Identifying prevalent false claims.
  2. Keyword selection: Defining search terms related to these topics.
  3. Tweet download: Collecting tweets returned by the selected keywords between February and September 2020.
  4. Manual classification: Labeling tweets as "Misinformation" or "Not Misinformation."
Process of corpus construction for COVID-19 misinformation detection.

The resulting dataset consists of 730 unique tweets, with 456 labeled as misinformation and 274 as non-misinformation. The topics covered include lemon/bicarbonate cures, gargling with vinegar/salt, Vitamin C + Zinc, Ivermectin efficacy, and hot water as a preventative.

Feature Engineering and Modeling

Tweets were preprocessed to remove noise (accents, punctuation, stopwords, etc.). The study compared three textual representation techniques: Bag of Words (BoW), TF-IDF (with 1-gram to 3-gram ranges), and Word2Vec (Skip-Gram). Six algorithms were evaluated: LightGBM, SVM, Random Forest, Naive Bayes, AdaBoost, and Logistic Regression. Optimization was performed via grid search and 5-fold cross-validation.

Results

The experimental results highlighted a clear advantage for traditional frequency-based representations over modern embeddings in this specific context.

ROC curves for various classifiers using TF-IDF representation.

As shown in the ROC curve analysis, the SVM model using TF-IDF (1-gram) and an RBF kernel emerged as the top performer. It achieved an AUC-ROC of 0.911, an F1-score of 89%, and an accuracy of 85%. Notably, the model reached a recall of 97%, which is critical for public health applications to ensure that the vast majority of misinformation is flagged, even at the cost of some false positives. In contrast, Word2Vec-based models performed significantly worse, with the best (LightGBM) reaching only a 76% F1-score.

Discussion and Conclusion

The superior performance of TF-IDF suggests that for COVID-19 misinformation, specific "trigger" keywords (e.g., "limão", "cura") provide a stronger signal than broader semantic context. The high recall of the SVM model makes it a viable candidate for real-world monitoring systems.

Future work will focus on expanding the dataset size to enable the use of deep learning architectures like BERT and conducting deeper error analysis to understand why certain tweets are misclassified. The dataset has been made publicly available on GitHub to encourage further research in Portuguese misinformation detection.

Summary updated

BibTeX
@inproceedings{mota2021deteccao,
  author = {Ana Alice Ximenes Mota and Wellington Franco and César Lincoln Cavalcante Mattos},
  title = {{Detecção de desinformação sobre Covid-19 no Twitter}},
  booktitle = {STIL},
  year = {2021},
  doi = {10.5753/stil.2021.17796},
}
Figure