KI2TE: Knowledge-Infused InterpreTable Embeddings for COVID-19 Misinformation Detection

William Shiao, Evangelos E. PapalexakisKnOD @ WWW 2021 · OpenReview

Abstract

The rapid spread of COVID-19 misinformation, or the "infodemic," necessitates automated detection systems that are not only accurate but also interpretable. We present KI²TE (Knowledge-Infused InterpreTable Embeddings), a framework that grounds news claims in a vast corpus of scientific literature (CORD-19). By transforming the relationship between claims and scientific papers into a distance matrix, KI²TE enables "white-box" classifiers to provide evidence-based explanations for their predictions. Our results demonstrate that KI²TE achieves performance comparable to state-of-the-art deep learning models while offering transparency by identifying the specific research papers that influence classification decisions.

Introduction

The COVID-19 pandemic was accompanied by a surge of misinformation regarding treatments, transmission, and vaccines. While deep learning models like BERT effectively identify fake news, they often function as "black boxes," providing little insight into the reasoning behind their classifications. This lack of transparency is a significant hurdle for trust and verification in sensitive domains like public health. Human fact-checking is accurate but slow, and knowledge graphs are difficult to maintain. KI²TE addresses this gap by using scientific literature as a dynamic ground truth, allowing for automated, interpretable misinformation detection that looks for substantive alignment with scientific consensus rather than just writing style.

Methods

The core of the KI²TE approach is a three-stage pipeline designed to bridge the gap between high-performance embeddings and interpretable logic:

  1. Shared Embedding Space: News articles and reference documents from the CORD-19 dataset are converted into mathematical vectors using embedding models such as Sentence-BERT (S-BERT) or FastText.
  2. Similarity Matrix Construction: The system calculates the pairwise distance between a news article and every reference document in the corpus. This creates a distance matrix $M$, where each feature represents the article's similarity to a specific scientific paper.
  3. Interpretable Classification: These similarity features are fed into a transparent classifier, such as Random Forest (RF) or Logistic Regression (LR).
The KI²TE architecture: from raw text to interpretable classification via a shared embedding space and distance matrix.

Because the features are tied to specific documents, the model can explain a "Fake" label by showing which scientific papers were most influential in the decision.

Results and Key Insights

The model was evaluated using several datasets, including CORD-19 (200,000+ papers), human-labeled claims from the Google FactCheck API, and the MM-COVID multilingual benchmark.

KI²TE demonstrates competitive performance against standard BERT-based models. In specific domains like vaccine and transmission news, the model leverages scientific consensus to achieve high accuracy and F1 scores.

Performance comparison of various model configurations across Vaccine and Transmission news datasets.

Key findings include:

  • Competitive Accuracy: The BERT + KI²TE + LR configuration reached an accuracy of 91.8% on the MM-COVID dataset.
  • Knowledge Scaling: Performance generally improves with the number of reference documents, though returns diminish after 1,000–2,000 documents, suggesting a representative subset of science is often sufficient to debunk most claims.
  • The Explainability Gap: The model performs best on scientific topics (Vaccine, Transmission) well-covered in literature. It successfully identifies top-contributing research papers for these claims, providing the "why" behind its decisions.

Discussion and Conclusion

While KI²TE provides robust interpretability, its primary challenge is scalability, as each reference document adds a feature to the classifier. Future work will focus on feature selection (e.g., Lasso) and clustering (e.g., k-means++) to manage large reference sets without losing breadth. By grounding truth in peer-reviewed science, KI²TE offers a scalable, transparent solution for combating misinformation in the digital age.

Summary updated

BibTeX
@inproceedings{shiao2021ki,
  author = {William Shiao and Evangelos E. Papalexakis},
  title = {{KI2TE: Knowledge-Infused InterpreTable Embeddings for COVID-19 Misinformation Detection}},
  booktitle = {KnOD @ WWW},
  year = {2021},
  url = {https://openreview.net/forum?id=XTP9shQrReY},
}
Figure