End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Models

Barry Menglong Yao, Aditya Shah, Lichao Sun, Jin-Hee Cho, Lifu HuangZenodo 2022 · DOI · OpenAlex

Abstract

We propose the end-to-end multimodal fact-checking and explanation generation, where the input is a claim and a large collection of web sources, including articles, images, videos, and tweets, and the goal is to assess the truthfulness of the claim by retrieving relevant evidences and predicting a truthfulness label (i.e., support, refute and not enough information), and to generate a rationalization statement to explain the reasoning and ruling process. To support this research, we construct MOCHEG, a large-scale dataset consisting of 21,184 claims where each claim is annotated with a truthfulness label and ruling statement, with 43,148 text evidences and 15,373 image evidences.
BibTeX
@inproceedings{yao2022end,
  author = {Barry Menglong Yao and Aditya Shah and Lichao Sun and Jin-Hee Cho and Lifu Huang},
  title = {{End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Models}},
  booktitle = {Zenodo},
  year = {2022},
  doi = {10.5281/zenodo.6653772},
}

Loading related content...

Figure