TL;DR
This paper introduces a novel computer vision system for automatically assessing physical activity in outdoor public spaces, moving beyond manual observation. The system demonstrates promising sample-level validity, accurately estimating sedentary behavior and moderate-to-vigorous physical activity (MVPA) proportions, which are key metrics for public health.
What Problem Does This Paper Address?
Traditional methods for assessing physical activity in public spaces, like parks and schoolyards, rely on direct observation. While valuable, these methods are labor-intensive, time-consuming, and often capture only snapshots of activity, limiting their ability to provide continuous monitoring or rapid feedback for interventions. Existing automated approaches have faced limitations, such as requiring participants to wear specific colored vests or being restricted to controlled indoor environments. This study aims to bridge this gap by developing and validating a computer vision system capable of automated, continuous, and ecological assessment of physical activity in open outdoor settings.
Methods
The core of this research lies in developing and evaluating computer vision algorithms for physical activity assessment. The system utilizes deep convolutional neural networks for action recognition to process video data from three cameras. These algorithms were trained to predict:
- The number of participants in different activity states (sedentary, light, moderate, vigorous).
- The proportion of participants in these states.
- Group-based metabolic equivalents of tasks (MET)-minutes.
The system was designed to be adaptable to outdoor environments, handling variations in lighting and camera angles, and importantly, did not require participants to wear special clothing. Criterion validity was assessed by comparing the computer vision algorithm's output against data from accelerometers worn by participants. Both second-level (1-second intervals) and minute-level aggregated data were analyzed.
Datasets
The study utilized video data collected from three digital cameras in an open outdoor area on the University of California, San Diego campus. This video data was synchronized with accelerometer data from nine participants. The total dataset comprised 6630 seconds of matched accelerometer and video data per camera. For training and testing the computer vision algorithms, blocks of 200 seconds of video were alternately assigned to a training dataset (N = 3830) and a testing dataset (N = 2800). No new benchmark datasets were introduced; rather, existing accelerometer data served as the ground truth.
Key Insights and Ideas
- Automated Observation is Feasible: The study demonstrates that computer vision can effectively automate the process of observing and quantifying physical activity in real-world outdoor settings, offering a significant advancement over manual methods.
- MVPA and Sedentary Behavior are Key: The algorithms showed strong validity for estimating sedentary behavior and moderate-to-vigorous physical activity (MVPA) proportions. This is particularly important as these are commonly used metrics in public health research and interventions.
- Combining Moderate and Vigorous Activity is Beneficial: While the algorithms had mixed results for individual moderate and vigorous activity levels, combining them into MVPA improved overall validity, highlighting the practical utility of this aggregated metric.
- Outdoor Adaptability is Crucial: The system's design to handle outdoor variability (lighting, camera angles) is a critical step towards broader applicability in diverse public spaces like parks and schoolyards.
- Sample-Level Validity is Achievable: The system demonstrated acceptable "sample-level" validity, meaning it can reliably estimate activity across a collection of time intervals, which is suitable for ecological assessments of settings.
Key Results
- Sedentary and MVPA: The number and proportion of participants in sedentary and MVPA categories showed small biases (within 20% of the criterion mean) and excellent intraclass correlation coefficients (ICCs) ranging from 0.82 to 0.98.
- Total MET-minutes: Total MET-minutes were slightly underestimated by 9.3-17.1%, with good ICCs (0.68–0.79).
- Light Activity: The algorithms showed poorer performance for light activity, with lower ICCs.
- Limits of Agreement: While sample-level validity was good, the limits of agreement for individual time intervals were wider than desired (outside the 30% criterion), indicating variability in bias across shorter time scales. This suggests that aggregation of data over longer periods (e.g., 5 minutes) might be necessary for more precise individual-level estimates.
Limitations and Future Work
The authors acknowledge several limitations:
- Limited Setting Diversity: The study was conducted in a single, open outdoor space with no obstructions, which limits generalizability to more complex environments with trees, playground equipment, or varied surfaces.
- Small Sample Size: The number of participants and observation days were relatively small.
- Prescribed Activities: Participants engaged in specific, prescribed activities, which may not fully reflect the spontaneity of real-world park use.
- Individual-Level Validity: While sample-level validity was good, the limits of agreement suggest caution when interpreting short-interval data at an individual level.
Future work should focus on:
- Testing in Diverse Settings: Evaluating the algorithms in a wider range of environments with varying demographics, group sizes, and physical features.
- Longer Duration Data Collection: To support aggregation over longer time intervals and improve individual-level validity.
- Investigating Predictors of Bias: Understanding factors that contribute to bias to inform data interpretation.
- Exploring Other Capabilities: Investigating whether the system can identify demographic information like gender, race/ethnicity, or age group, which is often captured by traditional observation methods.
- Comparison with Traditional Methods: Directly comparing the accuracy and efficiency of the computer vision system against established direct observation tools like SOPARC.
Summary updated