Beyond Text: Leveraging Multi-Task Learning and Cognitive Appraisal Theory for Post-Purchase Intention Analysis

Gerard Yeo, Shaz Furniturewala, Kokil JaidkaACL 2024 · DOI

Abstract

Predicting post-purchase consumer behavior—such as the intent to repurchase or recommend a product—is a critical challenge for e-commerce. While standard Natural Language Processing (NLP) models excel at sentiment classification, they often fail to capture the underlying psychological drivers of behavior. This paper explores the integration of Cognitive Appraisal Theory (CAT) into machine learning frameworks. By modeling the cognitive evaluations (appraisals) and emotions that precede behavior, we demonstrate that psychological constructs provide essential context that raw text lacks, leading to superior predictive performance.

Introduction

Supervised machine learning models for predicting user behavior typically underperform compared to standard text classification tasks. Most NLP models treat text as a standalone signal, ignoring the internal psychological processes of the author. In psychology, Cognitive Appraisal Theory (CAT) posits that emotions are the result of specific evaluations of a situation (e.g., "Is this product helping me reach my goal?" or "Who is responsible for this failure?"). For instance, a negative experience might only lead to a decision not to repurchase if the customer appraises the situation as "unfair" and the provider as "accountable." By ignoring these appraisals, current models miss the causal chain from experience to decision.

Methodology

The researchers evaluated several model architectures to determine how psychological data enhances behavioral prediction:

  1. Baseline Models: Separate models predicting behavior from text (using BERT), appraisal ratings, or emotion ratings.
  2. Constrained Models: A "bottleneck" approach where BERT predicts appraisals or emotions first, which are then used to predict behavior.
  3. Multi-modal Models: Architectures that concatenate text embeddings from BERT with human-annotated appraisal and emotion ratings.
  4. Multi-task & Theoretical Models: Models trained to predict appraisals, emotions, and behavior simultaneously. The "Theoretical Model" specifically mimics the psychological flow: Text → Appraisals → Emotions → Behavior.
Overview of the evaluated model architectures, including baseline, constrained, multi-modal, and multi-task approaches.

Dataset

The study utilized the PEACE-Reviews Dataset, containing 1,400 author-annotated product reviews. Unlike typical datasets where third-party labellers guess sentiment, this dataset includes original author annotations for 20 appraisal dimensions (e.g., Pleasantness, Goal Conduciveness), 8 emotion intensities (e.g., Anger, Joy), and Post-Consumption Behaviors (PCBs) including the likelihood to repurchase and promote.

Key Results and Insights

The experimental results validate the hypothesis that psychological appraisals are high-signal features for behavioral prediction. Models using only appraisal ratings outperformed text-only models, suggesting that how a person evaluates their experience is a more stable predictor of behavior than the specific vocabulary used in a review.

Performance comparison of different model architectures across Intent to Repurchase and Intent to Promote tasks.

Key findings include:

  • Top Performance: The Appraisals → PCB baseline achieved the highest accuracy for both "Intention to Repurchase" (73.4%) and "Intention to Promote" (75.9%).
  • Text vs. Psychology: The baseline text-only model (BERT) reached 70.1% and 72.4% respectively, showing a clear 3-4% gain when psychological appraisals were utilized.
  • Behavioral Drivers: "Intention to Promote" (word-of-mouth) is heavily driven by affective, sentiment-laden language. Conversely, "Intention to Repurchase" is more complex, relying on a blend of cognitive appraisals like "consistency" and "importance."
  • Theory-Informed Architecture: The "Theoretical Model," which follows the causal chain of psychology, performed particularly well for predicting the intention to promote, validating the utility of hard-coding psychological theory into neural network structures.

Conclusion

This research demonstrates that integrating psychological theory into NLP frameworks significantly enhances the prediction of consumer behavior. While the current study is limited by a relatively small dataset of 1,400 reviews focused on high-stakes purchases, it opens new avenues for explainable AI. Future work will explore the ability of Large Language Models (LLMs) to "zero-shot" these appraisals from raw text, potentially bridging the data gap for datasets lacking rich psychological annotations.

Summary updated

BibTeX
@inproceedings{yeo2024beyond,
  author = {Gerard Yeo and Shaz Furniturewala and Kokil Jaidka},
  title = {{Beyond Text: Leveraging Multi-Task Learning and Cognitive Appraisal Theory for Post-Purchase Intention Analysis}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-acl.734},
}
Figure