Barriers and facilitators of obtaining SDoH of patients with cancer through the EHR using natural language processing technology: A qualitative study (Preprint)

Jordan M. Alpert, Julia Kim, Cara McDonnell, Yi Guo, Thomas J. George, Jiang Bian, Yonghui Wu2022 · DOI

Introduction

Social determinants of health (SDoH)—including housing stability, food security, and social support—account for up to 90% of modifiable health outcomes. Despite their clinical significance, SDoH data is frequently missing from structured fields in Electronic Health Records (EHRs). Instead, this vital information is often "hidden" within unstructured clinical narratives. While Natural Language Processing (NLP) offers a path to automate the extraction of this data, successful implementation requires a deep understanding of how stakeholders document and utilize social context.

Methodology

This qualitative study employed semi-structured interviews with 16 key stakeholders at the University of Florida Health to explore the feasibility and challenges of NLP-driven SDoH extraction. The participant pool included:

  • Oncologists: Primary creators of clinical documentation.
  • Data Analysts: End-users who leverage SDoH data for health services research.
  • Patient Navigators: Professionals who often maintain high-trust relationships with patients.
  • Citizen Scientists: Community members providing the patient perspective.

The research team utilized a grounded theory approach, analyzing 160 pages of interview transcripts via the constant comparative method to identify recurring themes and barriers.

Key Findings

The study identified several critical barriers to effective SDoH documentation and extraction:

1. The Usability Gap

Clinicians reported that entering discrete SDoH data into "smart forms" is often cumbersome, sometimes requiring up to 15 clicks for a single variable. Consequently, providers default to free-text notes, which preserve narrative nuance but complicate large-scale data analysis. Previous data from the OneFlorida Clinical Research Consortium supports this, showing that standardized Z-codes for SDoH were used in only approximately 2% of patient records.

2. The Nuance of Disclosure

Patient navigators often uncover more sensitive SDoH information (e.g., financial strain or domestic issues) than physicians. This suggests that NLP systems must analyze a broad range of documentation, including nursing notes and secure patient messaging (like MyChart), to capture a complete social profile.

3. Risks of Misinterpretation

Citizen scientists highlighted the risk of NLP miscoding. For instance, a patient expressing concern about a "paranoid neighbor" could be incorrectly flagged with a psychiatric diagnosis rather than a social safety concern. Ensuring high validation standards is essential to prevent such clinical inaccuracies.

4. Limitations of Current Proxies

Data analysts currently rely heavily on zip codes as a proxy for socioeconomic status. However, participants noted that this is a "blunt instrument" that fails to capture individual-level factors like personal income or specific educational barriers.

Actionable Integration

Stakeholders emphasized that NLP should do more than just extract data; it should make it actionable. Participants expressed a desire for integrated dashboards that provide a "social risk score" or summary before patient encounters, allowing for more personalized care delivery. The goal is to transform messy clinical narratives into structured, actionable medical insights.

Conclusion and Future Directions

Extracting SDoH from EHRs is viewed as essential for advancing health equity and outcomes research. While NLP is a promising solution, its development must account for the informal ways SDoH is documented and the specific needs of those using the data. Future work should focus on normalizing extracted text into common data models (like PCORnet or OMOP) and exploring patient-facing tools for direct SDoH entry to reduce the documentation burden on clinicians.

Summary updated

Abstract

<sec> <title>BACKGROUND</title> Social determinants of health (SDoH), such as geographic neighborhoods, access to healthcare, education, and social structure are important factors affecting people’s health and health outcomes. SDoH of patients are scarcely documented in a discrete format in electronic health records (EHRs) but are often available in free-text clinical narratives such as physician notes. Innovative methods like natural language processing (NLP) are being developed to identify and extract SDoH from EHRs, but it is imperative that the input of key stakeholders is included as NLP systems are designed. </sec> <sec> <title>OBJECTIVE</title> Understand the feasibility, challenges, and benefits of developing an NLP system to uncover SDoH from clinical narratives by conducting interviews with key stakeholders: 1) clinicians, 2) data analysts, 3) citizen scientists and 4) patient navigators. </sec> <sec> <title>METHODS</title> Individuals who frequently work with SDoH data were invited to participate in in-depth, semi-structured interviews. All interviews were recorded and subsequently transcribed. After coding transcripts and developing a codebook, the constant comparative method was used to generate themes. </sec> <sec> <title>RESULTS</title> A total of 16 participants were interviewed (five data analysts, four patient navigators, four physicians, and three citizen scientists). Two themes emerged related to collecting SDoH: 1) the importance of SDoH data and 2) SDoH arises during patient-clinician communication. The challenges of collecting SDoH data included: 1) informal communication and 2) the need for expertise and knowledge about SDoH. Ways of improving how SDoH data can be incorporated into health services research and patient care were to 1) empower patients and 2) make the data actionable. </sec> <sec> <title>CONCLUSIONS</title> Extracting SDoH from EHRs was considered valuable and necessary, but obstacles such as narrative data format can make the process difficult. NLP can be a potential solution, but as the technology is developed, it is important to consider how key stakeholders document SDoH, apply the NLP systems, and use the extracted SDoH in health outcome studies.natural language processing, qualitative, social determinants of health, electronic health records </sec>
BibTeX
@misc{alpert2022barriers,
  author = {Jordan M. Alpert and Julia Kim and Cara McDonnell and Yi Guo and Thomas J. George and Jiang Bian and Yonghui Wu},
  title = {{Barriers and facilitators of obtaining SDoH of patients with cancer through the EHR using natural language processing technology: A qualitative study (Preprint)}},
  year = {2022},
  doi = {10.2196/preprints.43059},
}
Figure