Abstract
Modeling persuasive language has the potential to better facilitate our decision-making processes. Despite its importance, computational modeling of persuasion is still in its infancy, largely due to the lack of benchmark datasets that can provide quantitative labels of persuasive strategies to expedite this line of research. To this end, we introduce a large-scale multi-domain text corpus for modeling persuasive strategies in good-faith text requests. Moreover, we design a hierarchical weakly-supervised latent variable model that can leverage partially labeled data to predict such associated persuasive strategies for each sentence, where the supervision comes from both the overall document-level labels and very limited sentence-level labels. Experimental results showed that our proposed method outperformed existing semi-supervised baselines significantly.
Introduction: The Challenge of Modeling Persuasion
Persuasion is a cornerstone of human communication, yet it remains a "black box" for computational linguistics. While we can often observe if a request succeeded (e.g., a loan was granted), it is much harder to quantify why it worked.
The primary challenges in this field include:
- Data Scarcity: Manually labeling every sentence in a request for specific tactics is prohibitively expensive.
- Domain Specificity: Persuasion tactics vary across platforms, making generalization difficult.
- Label Mismatch: There is a gap between "global" document-level outcomes and "local" sentence-level strategies.
Methodology: Weakly-Supervised Hierarchical Latent Variable Model (WS-VAE)
The authors propose a Weakly-Supervised Hierarchical Latent Variable Model (WS-VAE) to bridge the gap between sentence-level strategies and document-level success.
The architecture operates at two distinct levels:
- Sentence Level: A Variational Autoencoder (VAE) disentangles a sentence into two latent variables: $z$ (general content) and $y$ (a discrete persuasive strategy). This separation prevents the model from conflating what is being said with how it is being framed.
- Document Level: Latent variables from all sentences are fed into a predictor network (LSTM or BERT) to predict the final outcome (e.g., success or failure).
The "Weak Supervision" aspect allows the model to learn from unlabeled sentences by assigning them strategies that help the document-level predictor achieve the correct outcome.
Datasets and Taxonomy
The study introduces a large-scale multi-domain corpus across three platforms: Kiva (philanthropic lending), r/Random_Acts_Of_Pizza (RAOP), and r/borrow (personal loans). A unified taxonomy of eight generic persuasive strategies was developed to analyze these requests.
The final dataset comprises over 100,000 posts, with 1,200 documents manually annotated at the sentence level for evaluation.
Key Results and Insights
The WS-VAE-BERT model consistently outperformed standard supervised baselines, particularly when labeled data was scarce. In a "cold start" scenario with only 20 labeled documents, it achieved an F1 score of 41.0 on RAOP, significantly higher than a standard BERT model (30.6).
Other key insights include:
- Content vs. Strategy: While strategies are important, latent variable analysis suggests that the content ($z$) often carries more weight in determining a request's success. A well-explained reason is often more effective than mere politeness.
- Strategy Synergy: Certain strategies frequently appear together. For instance, on Kiva and RAOP, "Scarcity" (urgency) and "Evidence" (concrete facts) are highly correlated.
- Unified Effectiveness: The 8-strategy taxonomy proved effective across all three domains, suggesting these are fundamental pillars of human persuasion.
Conclusion and Future Work
While the model successfully identifies textual strategies, it currently assumes success depends solely on the text. Future work will aim to integrate external "audience factors"—such as requester reputation and social context—to provide a more holistic understanding of persuasion.
Summary updated