Building and benchmarking the motivated deception corpus: Improving the quality of deceptive text through gaming

Dan Barsever, Mark Steyvers, Emre NeftciBehavior 2022 · DOI

Abstract

When one studies fake news or false reviews, the first step to take is to find a corpus of text samples to work with. However, most deceptive corpora suffer from an intrinsic problem: there is little incentive for the providers of the deception to put their best effort, which risks lowering the quality and realism of the deception. The corpus described in this project, the Motivated Deception Corpus, aims to rectify this problem by gamifying the process of deceptive text collection. By having subjects play the game Two Truths and a Lie, and by rewarding those subjects that successfully fool their peers, we collect samples in such a way that the process itself improves the quality of the text. We have amassed a large corpus of deceptive text that is strongly incentivized to be convincing, and thus more reflective of real deceptive text. We provide results from several configurations of neural network prediction models to establish machine learning benchmarks on the data. This new corpus is demonstratively more challenging to classify with the current state of the art than previous corpora.

Introduction: The Incentive Gap in Deception Detection

Artificial Intelligence models trained to detect deception—such as fake news or fraudulent reviews—are only as good as the data they are fed. Currently, most datasets used for this purpose suffer from a significant "incentive gap." In typical research settings, participants are paid a flat fee to generate lies. Without a penalty for being caught or a reward for being convincing, these "low-stakes" lies often exhibit linguistic "tells"—such as simplified language or a lack of specific detail—that are absent in real-world scenarios where deceivers are highly motivated to succeed.

To bridge this gap, researchers developed the Motivated Deception Corpus (MDC). By gamifying the collection process, the MDC ensures that the resulting text reflects the effort and nuance characteristic of real-world deception.

Methodology: Gamifying Deception

The core of the MDC collection process is a gamified version of "Two Truths and a Lie." The study involved 160 participants who played 10 rounds across various topics, such as "Vacation" or "Exams."

The gamified data collection process for the Motivated Deception Corpus.

As shown in the figure above, the gameplay is divided into distinct phases:

  1. Writing Phase: Players write two true stories and one lie. To capture the "process" of lying, the researchers used the oTree platform to record every keystroke and millisecond-level timestamp.
  2. Judgement Phase: Peers attempt to identify the lie among the three stories.
  3. Incentive Structure: To simulate real-world stakes, players earned points (convertible to cash) for successfully fooling their peers and for correctly identifying the lies of others.

This competitive structure motivates participants to craft more sophisticated and convincing deceptive narratives, moving beyond the simplistic lies found in previous corpora.

Benchmarking and Results

The MDC contains 1,572 deceptive and 3,144 truthful samples. To evaluate its difficulty, the researchers benchmarked it against humans and state-of-the-art AI models, specifically BERT (Bidirectional Encoder Representations from Transformers).

AI Performance and Context

The results highlight the challenge posed by motivated lies. A BERT model that achieves an impressive 93.6% accuracy on the older Ott Deceptive Opinion Spam dataset managed only 57.8% accuracy on the MDC.

The study found that "context is king": the most successful AI models utilized "Next Sentence Prediction" to analyze story flow. When models were forced to evaluate a story in isolation, without the context of the accompanying "truths," performance plummeted. Furthermore, using Latent Semantic Analysis (LSA), the authors found that MDC lies were mathematically much closer to the LIAR dataset (real-world political lies) than traditional crowdsourced datasets.

Human and Behavioral Insights

Humans also struggled with the task. Even when motivated by financial rewards, participants correctly identified lies only 35.7% of the time, which is only marginally better than the 33.3% expected by pure chance.

Interestingly, the behavioral data—keystroke patterns and timing—did not provide a significant boost to detection accuracy. This suggests that the "cognitive load" of lying may not manifest in typing rhythms as clearly as previously hypothesized, at least when the writer is focused on being convincing.

Conclusion

The Motivated Deception Corpus demonstrates that when people are incentivized to win, their lies become linguistically indistinguishable from the truth for both humans and current AI. By gamifying data collection, researchers can create more robust benchmarks that better reflect the "wild" deception found in fake news and fraudulent reviews. Future work will focus on scaling this collection method and further exploring the behavioral nuances of how lies are constructed.

Summary updated

Abstract

When one studies fake news or false reviews, the first step to take is to find a corpus of text samples to work with. However, most deceptive corpora suffer from an intrinsic problem: there is little incentive for the providers of the deception to put their best effort, which risks lowering the quality and realism of the deception. The corpus described in this project, the Motivated Deception Corpus, aims to rectify this problem by gamifying the process of deceptive text collection. By having subjects play the game Two Truths and a Lie, and by rewarding those subjects that successfully fool their peers, we collect samples in such a way that the process itself improves the quality of the text. We have amassed a large corpus of deceptive text that is strongly incentivized to be convincing, and thus more reflective of real deceptive text. We provide results from several configurations of neural network prediction models to establish machine learning benchmarks on the data. This new corpus is demonstratively more challenging to classify with the current state of the art than previous corpora.
BibTeX
@inproceedings{barsever2022building,
  author = {Dan Barsever and Mark Steyvers and Emre Neftci},
  title = {{Building and benchmarking the motivated deception corpus: Improving the quality of deceptive text through gaming}},
  booktitle = {Behavior},
  year = {2022},
  doi = {10.3758/s13428-022-02028-7},
}
Figure