TL;DR
This paper explores using crowdsourcing via Amazon Mechanical Turk (MTurk) to gather culturally relevant health data for African American and Hispanic adults. This "seed data" is crucial for training recommender systems (RecSys) in mHealth apps, helping to overcome the "cold start problem" and mitigate algorithmic bias that can arise from using general population data.
What Problem Does This Paper Address?
Developing personalized mHealth interventions, particularly for disease prevention, faces a significant hurdle: the lack of publicly available, culturally relevant data for specific demographic groups, especially African Americans and Hispanic individuals. This data gap leads to a "cold start problem" for recommender systems (RecSys) within these apps. Without sufficient initial data aligned with the target users' priorities and cultural contexts, RecSys can provide irrelevant or even biased recommendations, potentially exacerbating existing health inequities. Traditional methods of data collection are often slow, expensive, and may not capture the nuances needed for effective personalization in diverse populations.
Methods
The researchers employed Amazon Mechanical Turk (MTurk) as a crowdsourcing platform to collect data from self-identified African American and Hispanic adults. Participants completed a web-based survey that included:
- Personal Profile: Demographics, family health history, and personal health history.
- Health Information Seeking Assessment: Questions about their experiences and frustrations in finding and using health information.
- Health Goal Selection: Participants chose their top three health goals related to preventable conditions.
- Article Rating: For each selected goal, participants reviewed and rated the top three relevant articles from the MyHealthfinder website based on importance, personal utility, and whether they would add it to their personal health library.
The study focused on collecting data to seed a RecSys, specifically analyzing the article ratings. The MTurk platform allowed for the recruitment of a large, diverse sample within a short timeframe and at a relatively low cost. Inclusion criteria ensured participants were self-identified as African American/Black or Hispanic/Latino/Latina/Latine, owned a smartphone, and were located in the US.
Datasets
The primary dataset introduced in this paper is a novel, crowdsourced dataset of health goals and article ratings from African American and Hispanic adults. This dataset was collected via MTurk and comprises responses from 985 participants. The articles rated were sourced from the MyHealthfinder website, a US Department of Health and Human Services resource. This dataset is intended to serve as "seed data" for training recommender systems, specifically addressing the lack of population-specific data for these demographic groups.
Key Insights and Ideas
- Crowdsourcing as a Solution to the Cold Start Problem: MTurk offers a practical and cost-effective method to generate the initial "seed data" needed to train recommender systems, thereby mitigating the cold start problem and improving early algorithm performance.
- Addressing Algorithmic Bias: By actively collecting data from the target populations (African American and Hispanic adults), the study aims to sidestep the algorithmic bias that can arise from training on general population data, which may not reflect the unique health priorities and cultural contexts of these groups.
- Cultural Relevance is Key for Personalization: The study highlights that effective personalization in mHealth requires data that is not only predictive but also culturally sensitive and relevant to the intended users.
- Efficiency and Cost-Effectiveness: Crowdsourcing via MTurk significantly reduces the time and cost associated with data collection compared to traditional methods, making it more feasible to gather data from underrepresented populations.
Key Results
- The MTurk crowdsourcing approach successfully generated 985 valid responses from 485 (49%) self-identified Hispanic and 500 (51%) self-identified non-Hispanic African American adults.
- Data collection was completed in 64 days at a cost of US $6.74 per respondent.
- Respondents rated a total of 92 unique articles, providing valuable data for seeding the RecSys.
- The top-rated articles for both groups, in terms of importance, included "Reduce Your Risk of Stroke," indicating a shared focus on major health threats. However, there were also distinct preferences, such as "Get Your Blood Pressure Checked" for Hispanic participants and "Quit Smoking" for Black participants, underscoring the need for tailored recommendations.
Limitations and Future Work
The authors acknowledge several limitations:
- Demographic Dissimilarity: While the study targeted specific populations, MTurk workers, on average, may be younger and more technologically literate than the general population, potentially leading to some demographic differences compared to the intended app user base.
- Generalizability of MTurk Workers: The study notes that MTurk workers might not perfectly represent the broader African American and Hispanic populations in all aspects.
Future work will involve studying the deployment of the HealthyMe/MiSalud RecSys trained on this seed data to quantify how demographic differences might affect the applicability of the preventative health information provided. The authors also suggest that the iterative learning nature of RecSys can help overcome initial mismatches, and that the generated seed data sets, if made publicly available, could significantly benefit other researchers and developers in creating population-specific health solutions.
Summary updated