Abstract
Background: The success of vaccination programs often depends on the effectiveness of the vaccine messages, particularly during emergencies such as the COVID-19 pandemic. The current suboptimal uptake of COVID-19 vaccines across many parts of the world highlights the tremendous challenges in overcoming vaccine hesitancy and refusal even in the context of a world-devastating pandemic. Methods: We conducted a randomized controlled trial in Hong Kong to evaluate the impact of seven vaccine messages on COVID-19 vaccine uptake (with the government slogan as the control). The participants included 127,000 individuals who googled COVID-19-related information during July-October 2021. Results: The impact of vaccine messages on uptake varied substantially over time and among different groups of users. For example, the message that emphasized the indirect protection of vaccination on family members (i) increased overall uptake by 30% (6-59%) in July but had no effect afterwards for English language users; and (ii) had no effect on overall uptake for Chinese language users throughout the study. Such volatility and heterogeneity in message effectiveness highlight the limitations of one-size-fits-all and static vaccine communication. Conclusions: Epidemic nowcasting should include real-time monitoring of vaccine hesitancy and message effectiveness, in order to adapt messaging appropriately. This dynamic dimension of surveillance has so far been underinvested.
Introduction
Public health messaging is frequently treated as a static, "set it and forget it" task. However, vaccine hesitancy exists on a complex spectrum rather than a binary "pro" or "anti" stance. While behavioral "nudges"—such as emphasizing family protection or financial incentives—are common tools to boost uptake, most evaluations of these messages occur at a single point in time.
This study argues that such a static approach is fundamentally flawed in the context of a fast-evolving crisis like the COVID-19 pandemic. Public sentiment is highly sensitive to local case counts, policy shifts, and cultural nuances. By examining the volatility (change over time) and heterogeneity (differences between subgroups) of message effectiveness, this research demonstrates that effective communication requires continuous, real-time monitoring.
Methods
The researchers conducted a large-scale Randomized Controlled Trial (RCT) in Hong Kong between July and October 2021, involving approximately 127,000 individuals. The study utilized Google Ads to deliver "nudges" to users searching for COVID-19-related information.
Experimental Design
When users entered specific keywords, they were randomized to see one of seven ad messages:
- Persuasion (Control): The standard government slogan.
- Comparison: Highlighting that Hong Kong lagged behind the UK/Singapore.
- Reopen: Emphasizing vaccination as a path to city reopening.
- Family: Focusing on protecting loved ones.
- Exemption: Highlighting the avoidance of mandatory testing.
- Lottery: Mentioning eligibility for vaccine-related prizes.
- Mortality: Highlighting global death tolls.
The study population was identified through a comprehensive set of keywords across three languages, as detailed in the table below:
Tracking and Data Integration
Unlike studies relying on self-reported "intent," this research tracked actual behavior. By matching anonymized IP addresses and timestamps with the official vaccination booking system in Hong Kong, the team identified confirmed bookings made within six hours of ad exposure. Data was stratified by language (Traditional Chinese, Simplified Chinese, and English) and age group.
Results: The Dynamic Nature of Nudges
The findings reveal that the effectiveness of vaccine nudges is both fleeting and highly specific to the demographic.
The "Expiration Date" of Nudges
A message's success can vanish rapidly. For English speakers, the "Family" message increased bookings by 30% in July but showed no significant effect just one month later. Similarly, for Traditional Chinese users aged 12–29, messages regarding family, lotteries, and reopening were highly effective in July (22% to 33% increases) but lost their impact or even reversed by August.
Cultural and Linguistic Divergence
Messages resonated differently across linguistic groups. The "Comparison" message (social proof) significantly boosted uptake for Traditional Chinese users in July (13% increase) but had no measurable effect on English or Simplified Chinese users.
The Backfire Effect
In some instances, messaging was counterproductive. The "Mortality" message, intended to emphasize the risk of the virus, actually reduced vaccine uptake among young Simplified Chinese users by 40% in July. This highlights the danger of untested or static messaging in sensitive populations.
Discussion and Conclusion
The study's results suggest that public health messaging should be treated as a form of "nowcasting." Just as epidemiologists track viral variants in real-time, public health departments should monitor "sentiment variants" to adapt communication strategies dynamically.
While this study faced limitations—such as a bias toward tech-savvy Google users and budget constraints that prevented granular real-time adjustments—it provides a proof-of-concept for future emergency responses. For future pandemics, the authors advocate for partnerships between public health departments and tech platforms to build bespoke, cost-effective surveillance systems that can pivot messaging as quickly as the public's mood shifts.
Summary updated