TL;DR
This paper is a response to a previous "Matters Arising" critique of their original study on wearable optical heart rate (HR) sensor accuracy across different skin tones. The authors defend their methodology, particularly regarding sample size and the use of the Fitzpatrick skin tone scale, while also acknowledging its limitations and suggesting avenues for future research.
What Problem Does This Paper Address?
The original study by Bent et al. aimed to investigate the potential for decreased accuracy in wearable optical HR sensors among individuals with darker skin tones. This is a critical issue because photoplethysmography (PPG), the technology behind most optical HR sensors, relies on light absorption, which can be influenced by melanin in the skin. The "Matters Arising" (MA) paper raised concerns about the original study's sample size and the suitability of the Fitzpatrick (FP) skin tone scale for categorizing skin tones. This reply directly addresses those concerns, defending the original study's design while also engaging with the critique to advance the field.
Methods
The authors defend their original study's methodology, which included:
- Analysis of Variance (ANOVA): Used to test for differences in mean HR accuracy between FP skin tone groups.
- Mixed Effects Regression Model: Employed to explore potential effects of wearable device type and activity type on accuracy.
- Power Analysis: Used to determine the necessary sample size for the ANOVA and mixed effects models, with a medium effect size of 0.3 chosen based on a pilot study.
- Fitzpatrick (FP) Skin Tone Scale: Used as the primary method for categorizing skin tones, with participants recruited to ensure an approximately equal distribution across the six FP groups.
- Objective Spectrophotometry: While the original study also used objective measurements (LinkSquare), the authors explain why they focused on visual assessments for their primary analysis.
Datasets
Not discussed in the paper. The study involved human participants and their physiological data.
Key Insights and Ideas
- Defending Sample Size: The authors justify their sample size based on achieving 80% power for their planned analyses, acknowledging that power analyses are imperfect but that their choices were statistically based.
- No Variance Differences: Contrary to potential assumptions, they found no significant differences in the variance of HR measurements or their errors across different skin tone groups.
- Covariate Considerations: They address the MA's suggestion of unmeasured covariates (arm hair, sweat, skin thickness), explaining the practical challenges of measuring them and the lack of literature support for their impact on PPG.
- FP Scale Nuances: While acknowledging the FP scale's limitations (visual assessment, not developed for the full spectrum), they argue it's still a valuable and accessible tool, especially when used with trained administrators and consistent conditions. They also highlight that some cited references questioning FP's correlation with skin color did not actually assess FP.
- Spectrophotometry vs. Visual Scales: They discuss the trade-offs between objective spectrophotometry (accurate but costly and requires specialized equipment) and subjective visual scales (accessible but prone to bias). They suggest that while spectrophotometry offers objectivity, its cost can be a barrier to entry for broader research.
- Call for Broader Application: The authors emphasize the need to expand this research framework to all devices used in clinical research across a wider variety of use cases to better capture real-world scenarios.
Key Results
- The original study was designed to have 80% power to detect a medium effect size (f² = 0.15) for differences in HR accuracy between FP skin tone groups.
- They recruited an approximately equal distribution of skin tones (7, 8, 10, 9, 9, and 10 participants for FP groups 1-6).
- No interaction effects were found between skin tone and activity level that could not be directly attributed to activity differences.
- No relationship was found between weight, BMI, body fat percentage, and HR accuracy, nor any interaction with skin tone.
- They found no differences in the variance of HR measurements or their errors across skin tone groups.
- Spectrometry measurements did not show differences across even the most divergent skin tone groups (FP1 vs. FP6), suggesting potential equipment error or other influencing factors not related to skin tone.
Limitations and Future Work
- Limitations of Visual Scales: The authors acknowledge that visual skin tone scales are imperfect and that a study can never definitively prove a null hypothesis.
- Spectrophotometry Limitations: While objective, spectrophotometry requires specialized equipment, is costly, and measurements can be affected by tissue composition beyond melanin.
- Future Work:
- Encourage more work examining wearable device accuracy across skin tones.
- Explore skin epidermal thickness measurement with appropriate optical equipment.
- Develop and establish best practices for spectrophotometry measures and potential sources of error.
- Broaden the application of their study framework to all devices used in clinical research under a wider variety of circumstances.
- Continue research as hardware and software evolve, understanding how software updates impact accuracy.
- Encourage wearables companies and researchers to use appropriately powered, IRB-approved studies to evaluate device accuracy across the full spectrum of skin tones.
Summary updated