What and Why? Towards Duo Explainable Fauxtography Detection Under Constrained Supervision
Lists
Only you see this
Abstract
Fauxtography is a category of multi-modal posts that spread misleading information on various big data online social platforms that generate billions of posts on a daily basis (e.g., Facebook, Twitter, Reddit). A fauxtography post usually consists of an image, a text description and comments from its readers. In this paper, we focus on explaining fauxtography posts by identifying <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">what</i> specific component and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">why</i> that component of a post leads to the fauxtography (i.e., duo explanations). This problem is motivated by the limitations of current fauxtography detection solutions that only focus on the detection but ignore the important <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">explanation</i> aspect of their results. Two critical challenges exist in solving our problem: i) it is difficult to accurately identify the “guilty” component of a fauxtography post given the fact that different components of the post and their associations could all lead to the fauxtography; ii) it is expensive and time-consuming to obtain a good training set with fine-grained labels of fauxtography posts in terms of explainability, making it challenging to develop fully supervised explainable solutions. To address the above challenges, we develop a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</b> uo <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Ex</b> plainable <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</b> auxtography Detection Framework under a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</b> onstrained Supervision (DExFC) to generate duo explanations from both content and comment parts of fauxtography posts. We evaluate the DExFC by creating real-world datasets from different online social media platforms (Twitter and Reddit). The results show that DExFC not only detects the fauxtography posts more accurately than the state-of-the-art solutions but also provides well-justified explanations to its results without the full supervision.
BibTeX
@article{kou2023what,
author = {Ziyi Kou and Daniel Yue Zhang and Lanyu Shang and Dong Wang},
title = {{What and Why? Towards Duo Explainable Fauxtography Detection Under Constrained Supervision}},
journal = {IEEE Trans. Big Data},
year = {2023},
url = {https://openreview.net/forum?id=8FqACg7N7l},
}Loading related content...