Introduction
In classical argumentation theory, a "good" argument is traditionally defined by its adherence to formal logic and factual evidence. However, real-world human communication often relies on a different persuasive tool: the personal anecdote. These "reports" of personal experience serve to build empathy, illustrate complex points, and foster inclusivity in public discourse. Despite their prevalence, the NLP community has historically focused on logical structures, leaving narrative evidence under-studied. This research bridges that gap by developing robust classifiers to identify personal stories across diverse argumentative domains.
Datasets and the "Conceptual Core"
To ensure the generalizability of their findings, the researchers analyzed three distinct datasets representing different rhetorical contexts:
- Regulation Room (RegRoom): Public comments from a U.S. e-rulemaking platform concerning debt collection regulations.
- Change My View (CMV): Persuasive Reddit threads where users attempt to change one another's opinions on various topics.
- Europolis: Transcribed and translated speeches from a transnational European political deliberative poll.
A key challenge was the variation in terminology across these sources; RegRoom and CMV were annotated for "testimony," while Europolis used the label "storytelling." However, the study hypothesized that these categories share a "conceptual core"—a set of universal linguistic markers that define a personal report regardless of the specific label used.
Methodology
The researchers framed the identification of these reports as a binary classification task. They evaluated several model architectures to determine the most effective approach for cross-domain detection:
- Feature-based: A Random Forest classifier utilizing 51 hand-crafted linguistic features, such as the frequency of first-person pronouns, past-tense verbs, and sentiment polarity.
- Neural Baselines: Standard Bag-of-Words (BoW) and Feedforward Neural Networks.
- Transformer-based: Fine-tuned BERT models, including domain-adapted versions pre-trained on parliamentary (Europarl) and social media (Reddit) corpora.
To analyze the drivers of model performance, the team employed linear regression for "performance mining," statistically isolating the impact of the test corpus, training setup, and model choice on the final F1-scores.
Key Results and Performance Analysis
The results confirmed that Transformer-based models significantly outperform traditional baselines. BERT models proved surprisingly robust, with the domain-adapted "BERT-mixed" model—trained on a combination of all three datasets—achieving the best overall performance. This "all" training setup consistently outperformed in-domain training, suggesting that exposure to diverse narrative styles helps the model learn the essence of a "report" rather than over-fitting to domain-specific quirks.
Specifically, the best models achieved macro F1-scores of 0.76 on Europolis, 0.85 on CMV, and 0.94 on RegRoom. Performance mining revealed that the most predictive linguistic fingerprints for these reports are the use of first-person pronouns ("I", "me"), past-tense verbs, and longer contribution lengths.
Discussion and Future Work
A pilot case study conducted as part of this research found that arguments containing personal reports were perceived as having higher quality, specifically scoring better in appropriateness, emotional appeal, and sufficiency. While stories increase the inclusivity of a debate, the authors caution that they can also serve as a vehicle for non-verifiable information or emotional propaganda.
Future research will focus on distinguishing between "individual" reports (unique personal stories) and "collective" reports (shared experiences) to see how each impacts deliberation quality. Additionally, the team aims to apply these classification techniques to detect AI-generated "false stories" in online discourse.
Summary updated