Illuminating the Unseen: Investigating the Context-induced Harms in Behavioral Sensing

Han Zhang, Vedant Das Swain, Leijie Wang, Nan Gao, Yilun Sheng, Xuhai Xu, Flora D. Salim, Koustuv Saha, Anind K. Dey, Jennifer MankoffarXiv 2024 · DOI · arXiv

Introduction

Behavioral sensing—the use of mobile and wearable sensors to track human behavior and well-being—is increasingly utilized in mental health and education. However, traditional fairness evaluations often focus on broad demographic categories like age or gender, overlooking more nuanced "context-induced harms." This research identifies two critical gaps: situated identity-based harm (biases against identities relevant to specific environments, such as first-generation students) and situation-based harm (performance drops caused by physical or technological settings, like room temperature or device type).

A Six-Step Framework for Responsible Sensing

To address these gaps, the researchers developed a human-centered, iterative framework designed to identify and mitigate harms throughout the sensing lifecycle. Unlike generic machine learning audits, this framework emphasizes user engagement and longitudinal monitoring.

The six-step framework for responsible deployment and harm evaluation, highlighting the iterative process of bias mitigation and inclusiveness improvement.

The framework consists of:

  1. Comprehensively understanding the context through user engagement.
  2. Establishing evaluation criteria using specific metrics like False Negative Rate disparities.
  3. Evaluating data inclusivity to ensure training sets reflect diverse contexts.
  4. Assessing algorithm sensitivity to contextual changes.
  5. Conducting harm evaluation by combining quantitative metrics with qualitative user feedback.
  6. Continuous maintenance to update models as environments evolve.

Case Studies and Methodology

The framework was validated through post-hoc audits on two real-world datasets:

  • GLOBEM: A four-year longitudinal dataset of ~500 college students tracking depression via passive sensing (steps, sleep, phone use).
  • En-Gage: A dataset of 23 high school students capturing physiological and environmental data (temperature, CO2) to predict classroom engagement.
A detailed outline of how the six-step framework was applied to the Depression Detection and Engagement Prediction case studies.

Key Insights: Uncovering "Unseen" Biases

The study revealed significant disparities that traditional audits miss:

  • The "Unseen" Bias: In depression detection, algorithms showed significant performance disparities for first-generation students and immigrants—groups rarely included in standard fairness checks.
  • Environmental Inequity: In the engagement study, models were significantly less accurate for students in uncomfortable thermal environments. For example, the Mean Squared Error (MSE) for emotional engagement was $0.822$ for students who felt too cold, compared to $0.631$ for those who were comfortable. This suggests that students with less access to climate-controlled environments may be unfairly penalized by automated monitoring.
  • The Fairness "Whack-a-Mole": Efforts to mitigate bias for one group (e.g., first-generation students) often introduced new biases against others (e.g., non-male or non-heterosexual participants). This "bias shift" highlights the complex trade-offs inherent in optimizing for fairness.

Conclusion and Future Work

While the framework successfully identifies context-induced harms, the researchers note limitations regarding data scarcity for small sensitive groups (e.g., students with disabilities) and the potential for harms arising from user interface (UI/UX) design. Future work will focus on validating the framework in "live" deployments and expanding its scope to include privacy and accountability dimensions in behavioral sensing.

Summary updated

Abstract

Behavioral sensing technologies are rapidly evolving across a range of well-being applications. Despite its potential, concerns about the responsible use of such technology are escalating. In response, recent research within the sensing technology has started to address these issues. While promising, they primarily focus on broad demographic categories and overlook more nuanced, context-specific identities. These approaches lack grounding within domain-specific harms that arise from deploying sensing technology in diverse social, environmental, and technological settings. Additionally, existing frameworks for evaluating harms are designed for a generic ML life cycle, and fail to adapt to the dynamic and longitudinal considerations for behavioral sensing technology. To address these gaps, we introduce a framework specifically designed for evaluating behavioral sensing technologies. This framework emphasizes a comprehensive understanding of context, particularly the situated identities of users and the deployment settings of the sensing technology. It also highlights the necessity for iterative harm mitigation and continuous maintenance to adapt to the evolving nature of technology and its use. We demonstrate the feasibility and generalizability of our framework through post-hoc evaluations on two real-world behavioral sensing studies conducted in different international contexts, involving varied population demographics and machine learning tasks. Our evaluations provide empirical evidence of both situated identity-based harm and more domain-specific harms, and discuss the trade-offs introduced by implementing bias mitigation techniques.
BibTeX
@misc{zhang2024illuminating,
  author = {Han Zhang and Vedant Das Swain and Leijie Wang and Nan Gao and Yilun Sheng and Xuhai Xu and Flora D. Salim and Koustuv Saha and Anind K. Dey and Jennifer Mankoff},
  title = {{Illuminating the Unseen: Investigating the Context-induced Harms in Behavioral Sensing}},
  year = {2024},
  eprint = {2404.14665},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.14665},
}
Figure