How Much Data Is Enough? A Reliable Methodology to Examine Long-Term Wearable Data Acquisition in Gait and Postural Sway

Brett M. Meyer, Paolo DePetrillo, Jaime Franco, Nicole Donahue, Samantha R. Fox, Aisling O'Leary, Bryn C. Loftness, Reed D. Gurchiek, Maura Buckley, Andrew J. Solomon, Sau Kuen Ng, Nick Cheney, Melissa Ceruolo, Ryan S. McGinnisSensors 2022 · OpenReview

TL;DR

This paper introduces a robust three-stage framework to determine the optimal duration for wearable sensor data collection in gait and postural sway analysis. Their findings suggest that while 2-3 days of data may suffice for capturing basic variability in gait and sway, longer periods (3-6 days) are often necessary for establishing meaningful correlations with patient-reported outcomes, highlighting a critical balance between data quantity, quality, and patient burden.

What Problem Does This Paper Address?

Wearable sensors offer a promising avenue for monitoring gait and balance impairments in real-world settings, moving beyond the limitations of lab-based assessments. However, a significant challenge in deploying these sensors for long-term studies is determining how much data is "enough." Collecting excessive data can burden participants and increase costs, while insufficient data may lead to unreliable measurements and missed insights. Previous research has explored wear duration for specific metrics, but lacked a standardized, comprehensive methodology that also considers the relationship between sensor data and clinical outcomes, particularly for complex measures like postural sway. This paper aims to fill that gap by proposing and validating a framework to systematically determine optimal wear durations for gait and postural sway analysis in a clinical population.

Methods

The authors propose a three-stage framework for analyzing wearable sensor data to determine optimal wear duration:

  1. Difference Testing: This stage uses statistical tests (e.g., rank sum tests) to identify if there are significant differences in gait and sway measures between various shorter wear durations (e.g., 1, 2, 3 days) and a longer baseline duration (e.g., 1 week). This helps understand when a measure stabilizes.
  2. Intra-Class Correlation (ICC): This stage assesses the reliability of measures across different wear durations by calculating ICC values. A high ICC (typically ≥ 0.70) indicates that the measure is consistent and reliable.
  3. Correlation to Established Clinical Measures: This crucial stage examines the association between sensor-derived gait and sway features and established patient-reported outcome measures (PROMs) like balance confidence and fatigue. This helps determine when the sensor data becomes clinically meaningful.

The study also employed a deep learning model for activity classification (walking vs. standing) and extracted numerous gait and postural sway features. A regression analysis was used to investigate factors influencing the required wear duration, specifically the number of daily observations and the variability of the measure.

Datasets

The study utilized wearable accelerometer data collected from 22 individuals with multiple sclerosis (PwMS). These participants wore sensors for 6 weeks, providing a substantial dataset for analysis. The data was collected using BioStamp nPoint® devices worn on the chest and thighs.

Key Insights and Ideas

  • A Multi-Stage Approach is Crucial: Relying on a single metric (like just ICC or just difference testing) can be misleading. Combining these with correlation to clinical outcomes provides a more holistic view of data sufficiency.
  • Gait vs. Sway Data Needs Differ: Postural sway measures appear to require longer monitoring periods than gait measures to achieve similar levels of reliability and clinical relevance. This is hypothesized to be due to the inherent variability and potentially fewer standing bouts available for analysis.
  • Clinical Relevance Dictates Duration: While 2-3 days of data might capture basic variability in gait and sway, achieving significant correlations with patient-reported outcomes often requires 3-6 days or even longer, especially for measures like fatigue.
  • Variability and Observation Frequency Matter: The regression analysis revealed that measures with higher inherent variability or fewer daily observations require longer wear durations. This suggests that understanding these characteristics a priori can help predict necessary data collection times.
  • Individualized Wear Duration: The study highlights that the "right" amount of data isn't universal; it depends on the specific measure being analyzed and the clinical question being asked.

Key Results

  • 2-3 Days for Basic Reliability: For many gait and sway features, 2 days of monitoring were sufficient to achieve strong ICCs and no significant differences compared to a 1-week baseline.
  • 3-6 Days for Clinical Correlations: To establish strong correlations with patient-reported measures like balance confidence (ABC) and fatigue (MFIS), longer durations were often needed. For instance, some gait asymmetry features correlated with ABC after just 1 day, while others, like sway distance and RMS AP for fatigue, required 2-4 days.
  • Specific Measures Need More Time: Certain complex gait stability measures (e.g., Lyapunov exponent) required up to a week of data to stabilize.
  • Regression Model: The regression analysis showed that variability (log-transformed CV) and the number of daily observations significantly predicted the required wear duration, explaining approximately 40% of the variance.

Limitations and Future Work

  • Sample Size and Diversity: The study acknowledges limitations due to a relatively small sample size (n=22) and a lack of demographic and geographic diversity among the PwMS. This may affect the generalizability of the findings.
  • Validity of Short Durations: The authors note that the optimal 2-3 day monitoring period's validity over longer timeframes (e.g., across seasons or disease progression) needs further investigation.
  • Prohibitive Long Baselines: For some populations, collecting a long baseline monitoring period might be impractical. In such cases, the regression approach to estimate necessary wear time is suggested as a more feasible alternative.
  • Future Work: The authors suggest conducting studies with larger, more diverse cohorts to confirm findings and explore the long-term validity of the proposed short monitoring periods. They also recommend investigating how to optimize protocols (e.g., by encouraging specific activities) to increase the number of daily observations for certain measures, potentially reducing required wear time.

Summary updated

Abstract

Wearable sensors facilitate the evaluation of gait and balance impairment in the free-living environment, often with observation periods spanning weeks, months, and even years. Data supporting the minimal duration of sensor wear, which is necessary to capture representative variability in impairment measures, are needed to balance patient burden, data quality, and study cost. Prior investigations have examined the duration required for resolving a variety of movement variables (e.g., gait speed, sit-to-stand tests), but these studies use differing methodologies and have only examined a small subset of potential measures of gait and balance impairment. Notably, postural sway measures have not yet been considered in these analyses. Here, we propose a three-level framework for examining this problem. Difference testing and intra-class correlations (ICC) are used to examine the agreement in features computed from potential wear durations (levels one and two). The association between features and established patient reported outcomes at each wear duration is also considered (level three) for determining the necessary wear duration. Utilizing wearable accelerometer data continuously collected from 22 persons with multiple sclerosis (PwMS) for 6 weeks, this framework suggests that 2 to 3 days of monitoring may be sufficient to capture most of the variability in gait and sway; however, longer periods (e.g., 3 to 6 days) may be needed to establish strong correlations to patient-reported clinical measures. Regression analysis indicates that the required wear duration depends on both the observation frequency and variability of the measure being considered. This approach provides a framework for evaluating wear duration as one aspect of the comprehensive assessment, which is necessary to ensure that wearable sensor-based methods for capturing gait and balance impairment in the free-living environment are fit for purpose.
BibTeX
@inproceedings{meyer2022how,
  author = {Brett M. Meyer and Paolo DePetrillo and Jaime Franco and Nicole Donahue and Samantha R. Fox and Aisling O'Leary and Bryn C. Loftness and Reed D. Gurchiek and Maura Buckley and Andrew J. Solomon and Sau Kuen Ng and Nick Cheney and Melissa Ceruolo and Ryan S. McGinnis},
  title = {{How Much Data Is Enough? A Reliable Methodology to Examine Long-Term Wearable Data Acquisition in Gait and Postural Sway}},
  booktitle = {Sensors},
  year = {2022},
  url = {https://openreview.net/forum?id=7dmqEZdEgF},
}
Figure