TL;DR
This paper explores using recommendation algorithms, common in e-commerce, to tailor health messages (specifically anti-smoking PSAs) to individual smokers. They found that their tailored algorithm predicted message persuasiveness more accurately than simpler methods and led to a greater likelihood of smokers evaluating PSAs favorably and attempting to quit smoking.
What Problem Does This Paper Address?
The effectiveness of health communication campaigns, particularly those aiming to change behavior like smoking cessation, often relies on tailoring messages to specific audiences. While message tailoring is known to enhance persuasion, traditional methods are resource-intensive. The authors investigate whether recommendation algorithms, successful in commercial settings for predicting user preferences, can be effectively adapted to predict individual responses to health messages and thereby improve campaign outcomes. They aim to bridge the gap between the proven effectiveness of tailoring and the practical challenges of implementing it at scale for complex health interventions.
Methods
The core of the study is the development and experimental testing of a message tailoring algorithm. This algorithm leverages a combination of:
- Collaborative Filtering: User-based (UBCF) and item-based (IBCF) approaches to identify similar users and similar messages based on past ratings.
- Matrix Factorization (MF): A more advanced technique to uncover latent features in user-item interactions.
- Individual-Level Data: Information about the smoker, such as stage of change, nicotine dependence, and demographics.
- Message-Level Data: Characteristics of the PSAs, including argument strength, narrative presence, and emotional content.
- Interactions: Combinations of individual and message features to capture more nuanced preferences.
This algorithm was trained on data from Wave 1 of a longitudinal study and then applied in Wave 2 to predict participants' persuasiveness ratings for PSAs they hadn't seen. The algorithm's performance was compared against two baseline methods: "Off the Shelf" (OTS), which involved random selection of PSAs, and "Best in Show" (BIS), which selected PSAs most preferred by the general smoker population.
Datasets
The study utilized data collected from a three-wave online experiment.
- Wave 1 (N=1,057): Baseline survey data, including demographic information, smoking history, nicotine dependence, and initial evaluations of eight PSAs. This data was used to train the tailoring algorithm.
- Wave 2 (N=675): Participants were randomly assigned to one of three conditions (OTS, BIS, Tailoring) and viewed four PSAs. They then evaluated these PSAs and reported their smoking cessation intentions.
- Wave 3 (N=525): A follow-up survey to assess self-reported quitting behavior.
The PSAs themselves were drawn from a larger archive of 200 anti-smoking PSAs, with 80 selected for this study based on varying levels of perceived effectiveness (PE). PE data for these PSAs was previously collected from independent samples of smokers.
Key Insights and Ideas
- Algorithmic Tailoring Mimics Social Influence: The study suggests that recommendation algorithms, by leveraging collaborative filtering, can mimic social influence processes. They recommend messages that similar others find persuasive, but without direct social interaction, offering a novel way to influence behavior.
- Behavioral Data is Key: The tailoring algorithm's success is attributed to its focus on actual choices and preferences (behavioral data) rather than solely relying on psychographic predictors, which is a departure from some traditional tailoring methods.
- Predictive Accuracy Matters for Impact: The superior predictive accuracy of the tailoring algorithm, as measured by lower RMSE, directly translated into more favorable PSA evaluations and a higher likelihood of quitting attempts. This highlights the critical link between accurate prediction and behavioral outcomes.
- Within-Participant Variability is Important: The tailoring algorithm's ability to reduce within-participant variability in PSA ratings suggests it better captures individual latent preferences, leading to more consistent and potentially more impactful recommendations.
Key Results
- Predictive Accuracy: The tailoring algorithm achieved significantly lower Root Mean Square Error (RMSE) in predicting participants' perceived effectiveness (PE) ratings compared to the "Best in Show" (BIS) method (.54 vs. .88 on average).
- Message Evaluation: Participants exposed to PSAs recommended by the tailoring algorithm evaluated them as significantly more persuasive than those in the "Off the Shelf" (OTS) condition. The difference between BIS and OTS was not statistically significant.
- Behavioral Impact: Smokers who viewed PSAs recommended by the tailoring algorithm were more likely to evaluate the PSAs favorably and attempt to quit smoking compared to those receiving randomly selected PSAs (OTS).
- No Significant Difference for BIS/OTS: The "Best in Show" (BIS) method did not show a significant advantage over the "Off the Shelf" (OTS) method in terms of message assessment or quitting behavior.
Limitations and Future Work
- Data Intensity: The authors acknowledge that their methodology is data-intensive, which might pose a challenge for practical implementation. They suggest that micro-targeting smaller, homogeneous groups could reduce data burdens.
- Algorithm Comparison: The study focused on comparing their developed tailoring algorithm against two standard approaches, rather than comparing different types of tailoring algorithms against each other.
- Small Effects: While statistically significant, the observed behavioral effects (quit attempts) were small. The authors note that this is understandable given the difficulty of changing established behaviors like smoking, especially among those who haven't yet quit.
- Future Research: The paper suggests exploring alternative collaborative filtering approaches beyond those tested and investigating the effectiveness of micro-targeting. Further research is also needed to explore how recommendation systems can be optimized for various health communication contexts and to understand the nuances of how these algorithms influence behavior.
Summary updated