TL;DR
This study quantifies the impact of Pokémon Go on physical activity using wearable sensor data and search engine logs from over 32,000 users. It reveals that playing Pokémon Go significantly increases physical activity, particularly for highly engaged users, and importantly, it reaches populations that are typically less active, unlike many existing health apps.
What Problem Does This Paper Address?
Despite widespread knowledge about the health benefits of physical activity, a significant portion of the population, both adults and children, fails to meet recommended activity levels. Numerous interventions aimed at increasing physical activity have proven ineffective, either by failing to reach inactive populations or by not being scalable across diverse demographics. The authors investigate whether novel mobile games, specifically Pokémon Go, which integrate gameplay with real-world movement, could offer a scalable and effective solution to this public health challenge.
Methods
The researchers employed a multi-faceted approach combining data from wearable devices and search engine queries.
- User Identification: They identified Pokémon Go players by analyzing Microsoft Bing search queries for terms related to "pokemon." Specifically, they manually annotated queries to distinguish between "experiential" (indicating active play) and "non-experiential" queries. A user was classified as a Pokémon Go player if they issued an experiential query.
- Physical Activity Measurement: Physical activity was measured using the 3-axis accelerometer and gyrometer data from Microsoft Band wearable devices, specifically by tracking the daily number of steps.
- Impact Analysis: They compared the physical activity levels of Pokémon Go users for 30 days before and 30 days after their first experiential query (denoted as $t_0$). This was done by comparing changes in daily steps.
- Control Group: A large control group of 50,000 US Microsoft Band users was used for comparison. To ensure fair comparison, a reference point ($t_0$) was randomly sampled for each control user, matching the distribution of $t_0$ from the Pokémon Go users.
- Dose-Response Analysis: The study investigated whether the level of engagement (measured by the number of experiential queries) correlated with the magnitude of physical activity increase.
- Demographic Analysis: The impact of Pokémon Go was examined across various demographic attributes, including age, gender, weight status (BMI), and prior activity levels.
- Comparison with Health Apps: The effect of Pokémon Go was also compared to that of four leading mobile health apps.
Datasets
- Microsoft Band Wearable Data: Daily step count data from over 32,000 Microsoft Band users over a 3-month period.
- Microsoft Bing Search Query Logs: Search queries from these users, specifically those mentioning "pokemon."
- User Self-Reported Demographics: Age, gender, height, and weight for the users.
The study did not introduce new benchmarks but rather utilized existing data to analyze the impact of a widely adopted application.
Key Insights and Ideas
- Gamification for Health: The study strongly suggests that well-designed games that require physical movement can be a powerful tool for promoting physical activity, even on a massive scale.
- Reaching the Unreached: A critical finding is that Pokémon Go effectively engages individuals with lower baseline physical activity levels, a demographic that traditional health interventions often struggle to reach.
- Engagement Matters: The degree of user engagement with the game, as indicated by the frequency of search queries related to gameplay, directly correlates with the increase in physical activity. This points to a dose-response relationship.
- Beyond Short-Term Boost: While the study focused on a 30-day period, the data suggests that even after the initial novelty wears off, more engaged users maintain a higher level of activity than they started with, hinting at potential for sustained behavior change.
Key Results
- Significant Activity Increase: Pokémon Go users showed significant increases in physical activity over a 30-day period following their first experiential query.
- Highly Engaged Users: Users making multiple search queries for game details (highly engaged) increased their activity by an average of 1473 steps per day, a 26% increase compared to their prior activity level.
- National Impact: It's estimated that Pokémon Go added a total of 144 billion steps to US physical activity within the first 30 days of its release.
- Broad Applicability: The positive impact on physical activity was observed across men and women of all ages, weight statuses, and prior activity levels.
- Reaching Low-Activity Populations: Pokémon Go users were less active than the average wearable user before playing the game, unlike users of other leading health apps who were already more active.
- Comparison with Health Apps: Pokémon Go users showed larger increases in physical activity compared to users of four leading mobile health apps.
- Meeting Guidelines: Highly engaged Pokémon Go users were almost 3 times more likely to meet official physical activity guidelines (8000 steps/day) after starting to play compared to before.
Limitations and Future Work
- Sample Bias: The study population is not a random sample of the US population, as it relies on users who own wearable devices and are willing to share their data. The sample was also predominantly male.
- Proxy for Engagement: The use of search queries as a proxy for Pokémon Go usage and engagement, while showing strong correlation, is still an indirect measure.
- Self-Reported Data: User demographic variables were self-reported.
- Short Follow-up Period: The primary follow-up period was limited to 30 days.
The authors emphasize the need for future studies to investigate the long-term effects of these applications on physical activity and behavior change. They also highlight the importance of understanding how to design games that sustain user engagement and how to integrate games with health interventions for maximum public health impact.
Summary updated