Difference Between Users and Nonusers of a Patient Portal in Health Behaviors and Outcomes: Retrospective Cohort Study

Jing Huang, Yong Chen, J. Richard Landis, Kevin MahoneyJournal of Medical Internet Research 2019 · DOI

Abstract

This paper contributes to the understanding of the impact of patient portal use on health outcomes and is the first study to identify a meaningful subgroup of patients' health behaviors that improved with portal use. These findings may encourage providers to promote portal use to improve patients' preventive health behaviors.

Introduction

Healthcare systems have invested heavily in patient portals to engage patients, yet evidence of their clinical impact remains thin. A major hurdle in this research is selection bias: patients who sign up for portals tend to be younger, wealthier, and more health-conscious. If a portal user has lower blood pressure, it is often unclear if the portal caused the improvement or if the user was already predisposed to better health management.

The myPennMedicine patient portal interface, an implementation of Epic’s MyChart used in the study.

This study sought to isolate the "portal effect" from these confounding factors to determine if digital tools like the myPennMedicine portal actually improve preventive care and chronic disease management.

Methods

The researchers conducted a retrospective cohort study of 10,000 patients aged 50 and older from the University of Pennsylvania Health System (UPHS). Data was sourced from the Penn Data Store (PDS), aggregating electronic health records (EHR) from 2014 to 2016.

To address selection bias, the study employed cardinality matching. Unlike traditional propensity score matching, which often discards significant amounts of data, cardinality matching uses integer programming to maximize the number of matched pairs while ensuring the two groups are perfectly balanced across key covariates, including age, income, race, insurance type, and provider characteristics. A novel bootstrapping approach was then used to accurately estimate the variance and statistical significance of the findings.

Key Results

After matching 3,465 pairs of users and nonusers to ensure a fair comparison, the study revealed a clear distinction between behavioral changes and clinical outcomes.

Significant Gains in Preventive Behaviors

Portal users were significantly more likely to engage in preventive care activities. The study found:

  • Flu shots: Odds Ratio (OR) of 1.58.
  • Lipid (LDL) tests: OR of 1.50.
  • Blood pressure checks: OR of 1.13.
  • Composite Prevention Score: Users scored 0.22 points higher on a 0-4 scale ($P < .001$).

No Significant Change in Chronic Outcomes

Despite improved preventive behaviors, there was no statistically significant difference in actual clinical markers over the study period:

  • Systolic Blood Pressure: Mean difference of -1.19 mm Hg ($P = .06$).
  • LDL Levels: Mean difference of -2.34 mg/dL ($P = .50$).
  • Chronic Conditions: No significant association was found with diabetes or hypertension status.

Discussion and Insights

The findings suggest that patient portals act effectively as "nudge" tools. They simplify discrete preventive actions, such as viewing a reminder for a flu shot or booking a lipid screen. However, the lack of impact on chronic outcomes suggests that while a portal is an excellent communication and scheduling tool, it is not a substitute for the complex lifestyle changes and clinical interventions required to manage long-term disease.

Furthermore, the data highlighted a persistent "digital divide." Before matching, portal users were significantly younger (63 vs. 66 years), more likely to be white (73% vs. 53%), and had higher annual incomes (~$74k vs. ~$63k) than nonusers. This suggests that while portals are effective, their benefits may not be reaching all demographic groups equally.

Limitations and Future Work

The study focused on a relatively healthy cohort aged 50+, and the two-year window may have been too short to observe changes in chronic health markers. Future research should utilize longitudinal studies to track health trajectories over longer periods and investigate "dose-response" relationships based on the frequency and type of portal interactions.

Summary updated

Abstract

This paper contributes to the understanding of the impact of patient portal use on health outcomes and is the first study to identify a meaningful subgroup of patients' health behaviors that improved with portal use. These findings may encourage providers to promote portal use to improve patients' preventive health behaviors.
BibTeX
@article{huang2019difference,
  author = {Jing Huang and Yong Chen and J. Richard Landis and Kevin Mahoney},
  title = {{Difference Between Users and Nonusers of a Patient Portal in Health Behaviors and Outcomes: Retrospective Cohort Study}},
  journal = {Journal of Medical Internet Research},
  year = {2019},
  doi = {10.2196/13146},
}
Figure