Receiving information on machine learning-based clinical decision support systems in psychiatric services may increase patient trust in these systems: A randomised survey experiment

Erik Perfalk, Martin Bernstorff, Andreas Aalkjær Danielsen, Søren Dinesen ØstergaardmedRxiv 2024 · DOI

This study, conducted by researchers at Aarhus University, explores the intersection of artificial intelligence and patient-clinician dynamics. As machine learning (ML) begins to assist in psychiatric diagnoses and treatment plans, the "black box" nature of these tools risks alienating patients. This paper demonstrates that providing even a brief, four-slide explanation of how ML works can measurably increase patient trust and decrease skepticism.

TL;DR

A randomized survey experiment of nearly 1,000 psychiatric patients found that providing basic educational information about machine learning-based decision support systems significantly increases patient trust and reduces distrust. The findings suggest that "explainability"—transparency regarding how models reach their conclusions—is a critical prerequisite for the successful implementation of AI in mental healthcare.

What Problem Does This Paper Address?

Machine learning models are increasingly capable of predicting clinical outcomes, such as suicide risk or the likelihood of needing mechanical restraint. However, psychiatry relies heavily on the therapeutic alliance—the trust between patient and provider. If patients perceive ML as an opaque, impersonal, or error-prone "third party" in their care, that alliance could crumble.

Prior research on AI trust has largely focused on the general public or general hospital settings. This paper addresses a critical gap: the specific attitudes of patients currently receiving psychiatric treatment, a group that may already experience lower levels of institutional trust. The authors sought to determine if a simple "information intervention" could mitigate "algorithm aversion."

Methods

The researchers conducted an online randomized survey experiment within the Psychiatric Services of the Central Denmark Region.

  • Participants: 5,800 patients were invited via a secure government mailing system; 992 completed the study.
  • The "Click-Time" Randomization: In a clever workaround for software limitations, participants were randomized into three arms based on the exact millisecond they accessed the survey link.
  • The Arms:
    1. Intervention: Received a visual and text-based pamphlet explaining how ML-based clinical decision support works.
  • Active Control: Received a pamphlet explaining standard clinical decision-making without ML.
  • Blank Control: Received no information pamphlet.
  • Measurement: After the intervention, all groups answered a questionnaire regarding trust (e.g., "I feel safe...") and distrust (e.g., "I am concerned about errors..."). The researchers used Principal Component Analysis (PCA) to group these responses into robust "Trust" and "Distrust" scores.

Datasets

The study did not use a traditional ML training dataset. Instead, it generated a primary dataset of survey responses from 992 psychiatric patients.

  • EHR Linkage: These survey results were linked to the patients' Electronic Health Records (EHR) to include objective clinical data, such as ICD-10 diagnoses (e.g., psychotic, mood, or personality disorders) and history of contact with psychiatric services. This allowed the researchers to ensure the intervention effect was consistent across different diagnostic groups.

Key Insights and Ideas

  • Explainability is Non-Negotiable: The highest-scoring item across all groups was the importance of receiving an explanation for why a model recommends a specific treatment (median score of 10/10). Patients aren't just looking for accuracy; they want to understand the "why."
  • Specific vs. General Information: The "Active Control" group (who read about standard care) showed no increase in trust toward ML. This indicates that trust isn't built by simply being "informative" in general; the information must specifically demystify the technology in question.
  • The "Algorithm Aversion" Buffer: While the effect size was small (Cohen’s D ≈ 0.2), the fact that a single, brief exposure to information moved the needle suggests that patients are open to these tools if they are not kept in the dark.
  • Baseline Openness: Contrary to fears that psychiatric patients might be uniquely hostile to AI, their baseline trust in ML was roughly equivalent to their trust in other public institutions like the police or the judicial system.

Key Results

  • Trust Increase: The intervention group showed a 5% higher trust score compared to the active control (p=0.009) and a 4% increase compared to the blank control (p=0.015).
  • Distrust Decrease: The intervention significantly reduced distrust scores by 3% compared to the active control (p=0.021) and 4% compared to the blank control (p=0.022).
  • Consistency: These effects remained significant even when the data was stratified by age, sex, and diagnosis, suggesting the intervention is broadly applicable across the psychiatric population.

Limitations and Future Work

The authors acknowledge several limitations:

  • Selection Bias: The 17% response rate suggests that the participants might be more tech-savvy or "pro-science" than the average patient.
  • Short-term Impact: Trust was measured immediately after the intervention. It is unclear if this trust persists over weeks or months of actual clinical use.
  • Geographic Specificity: Denmark is one of the most digitalized nations in the world. The "baseline" trust found here might be higher than in regions with less digital integration.
  • Future Directions: The authors suggest that future studies should test more "comprehensive and repeated" interventions—perhaps integrating ML explanations directly into the clinical workflow rather than a one-off pamphlet.

Summary updated

Abstract

Abstract Background Clinical decision support systems based on machine learning (ML) models are emerging within psychiatry. If patients do not trust this technology, its implementation may disrupt the patient-clinician relationship. Therefore, we examined whether receiving basic information about ML-based clinical decision support systems increased trust in them. Methods We conducted an online randomised survey experiment among patients receiving treatment in the Psychiatric Services of the Central Denmark Region. The participants were randomised to one of three arms, receiving different types of information: Intervention = information on clinical decision making supported by an ML model; Active control = information on a standard clinical decision process without ML-support; Blank control = no information. The participants were unaware of the randomization and the experiment. Subsequently, the participants were asked about different aspects of trust/distrust in ML-based clinical decision support systems. The effect of the intervention was assessed by comparing pairwise comparisons between all arms on component scores of trust and distrust. Findings Out of 5800 invitees, 992 completed the survey experiment. The intervention increased trust in ML-based clinical decision support systems when compared to the active control (mean absolute difference in trust: 5% [95%CI: 1%;9%], p-value= 0·009) and the blank control arm (mean absolute difference in trust: 4% [1%;8%], p-value=0·015). Similarly, the intervention significantly reduced distrust in ML-based clinical decision support systems when compared to the active control (mean absolute difference in distrust -3%[-5%; -1%], p-value=0·021) and the blank control arm (mean absolute difference in distrust -4% [-8%; -1%], p-value=0·022). For both trust and distrust, there were no material or statistically significant differences between the active and the blank control arms. Interpretation Receiving information on ML-based clinical decision support systems in hospital psychiatry may increase patient trust in such systems. Hence, implementation of this technology could ideally be accompanied by information to patients. Funding None. Research in context Evidence before this study Clinical decision support systems based on machine learning (ML) models are emerging within psychiatry. However, if patients do not trust this technology, its implementation may disrupt the patient-clinician relationship. Unfortunately, there is only little knowledge on opinions on ML models as decision support among patients receiving treatment in psychiatric services. Also, it remains unknown whether receiving basic information about ML-based clinical decision support systems increases patients’ trust in them. We searched PubMed on Sep 12, 2023, with the terms “((survey) OR (survey experiment)) AND (patients) AND ((opinions) OR (attitudes) OR (trust)) AND ((machine learning) OR (artificial intelligence)) AND ((Psychiatry) OR (Mental Disorders) OR (Mental Health))” with no language restrictions. This yielded a total of 73 records, none of which surveyed a patient population from psychiatric services. Only two studies were directly relevant for the topic at hand. One surveyed patients from a general hospital system in the United States about the use of ML-based prediction of suicide risk based on electronic health record data. The results showed that patients were generally supportive of this data use if based on consent and if there was an opportunity to opt out. The other study surveyed women from the general population about their opinion on the use of artificial intelligence (AI)-based technologies in mental healthcare. The results showed that the respondents were generally open towards such technologies but concerned about potential (medical harm) and inappropriate data sharing. Furthermore, the respondents identified explainability, i.e., understanding which information drives AI predictions, as being of particular importance. Added value of this study To the best of our knowledge, this is the first study to investigate opinions on ML-based clinical decision-support systems among patients receiving treatment in psychiatric services. On average, patients were open towards the use of ML-based clinical decision-support systems in psychiatry. Furthermore, the results suggest that providing basic information about this technology seems to increase patient trust in it, albeit with a small effect size. Finally, the results support prior reports on the importance of explainability for acceptance. Implications of all the available evidence Receiving information on ML-based clinical decision support systems in hospital psychiatry, including how they work (explainability), may increase patient trust in such systems. Hence, successful implementation of this technology likely requires information of patients.
BibTeX
@misc{perfalk2024receiving,
  author = {Erik Perfalk and Martin Bernstorff and Andreas Aalkjær Danielsen and Søren Dinesen Østergaard},
  title = {{Receiving information on machine learning-based clinical decision support systems in psychiatric services may increase patient trust in these systems: A randomised survey experiment}},
  year = {2024},
  doi = {10.1101/2024.04.11.24305655},
  howpublished = {medRxiv},
}
Figure