A Medical Data Reliability Assessment Model

Bandar Alhaqbani, Audun Jøsang, Colin FidgeJournal of Theoretical and Applied Electronic Commerce Research 2009 · DOI

Abstract: There is currently a strong focus worldwide on the potential of large-scale Electronic Health Record (EHR) systems to cut costs and improve patient outcomes through increased efficiency. This is accomplished by aggregating medical data from isolated Electronic Medical Record databases maintained by different healthcare providers. Concerns about the privacy and reliability of Electronic Health Records are crucial to healthcare service consumers. Traditional security mechanisms are designed to satisfy confidentiality, integrity, and availability requirements, but they fail to provide a measurement tool for data reliability from a data entry perspective. In this paper, we introduce a Medical Data Reliability Assessment (MDRA) service model to assess the reliability of medical data by evaluating the trustworthiness of its sources, usually the healthcare provider which created the data and the medical practitioner who diagnosed the patient and authorised entry of this data into the patient's medical record. The result is then expressed by manipulating health record metadata to alert medical practitioners relying on the information to possible reliability problems.

Introduction

As healthcare systems transition toward large-scale Electronic Health Record (EHR) aggregation, data is increasingly pulled from a fragmented landscape of hospitals, private clinics, and individual practitioners. While traditional cybersecurity frameworks focus on the "CIA triad"—Confidentiality, Integrity, and Availability—these measures primarily protect data from unauthorized access or tampering after it has been recorded. They do not address the fundamental issue of data quality at the point of entry.

If a practitioner has a history of clinical errors, or a facility is known for inconsistent record-keeping, current EHR systems lack the formal mechanisms to distinguish that input from high-quality data. This paper proposes a Medical Data Reliability Assessment (MDRA) model to bridge this gap, providing a quantitative measure of data reliability based on the trustworthiness of its source.

The MDRA Framework

The MDRA service acts as a middleware layer between the EHR system and various trust authorities. Its primary function is to evaluate the reliability of specific data fields by analyzing the reputation of the healthcare provider and the individual practitioner responsible for the entry.

The MDRA service architecture and its interactions with healthcare authorities and reputation centers.

As shown in the architecture, the MDRA service (1) receives data from a provider, (2) queries a Healthcare Authority for formal credentials, and (3) consults a Reputation Centre for community-based feedback. By fusing these diverse information sources, the system generates a reliability score that is attached to the record's metadata.

Mathematical Foundations

The model is built upon three core mathematical pillars:

  1. Beta Reputation System: This handles binary feedback (e.g., successful vs. unsuccessful interactions) to predict future behavior.
  2. Dirichlet Reputation System: This manages multinomial ratings, such as "star ratings" or severity levels (e.g., Bad, Mediocre, Average, Good, Excellent), allowing for more nuanced assessments.
  3. Subjective Logic: This serves as the computational engine. Unlike standard logic, subjective logic explicitly incorporates uncertainty. It allows the MDRA to combine opinions from different sources while "discounting" them based on the system's inherent trust in the source itself.

Transitive and Derived Trust

A critical innovation of the MDRA model is its ability to handle complex trust chains. In a large-scale medical network, a specific hospital may not have a direct history with every visiting consultant. The model uses transitive trust to bridge these gaps.

A diagram illustrating how Alice can derive a trust level for David by aggregating the opinions of Bob and Claire.

If a hospital (Alice) trusts a national authority (Bob), and that authority trusts a specific doctor (David), Alice can mathematically derive a trust level for David. This ensures that even "new" entities in the system can be assigned a baseline reliability score based on their professional affiliations.

Case Study and Results

The authors validated the framework using a synthetic motivational scenario involving a patient misdiagnosed by a practitioner with a poor historical record. By applying the MDRA protocol, the service queried multiple sources and applied subjective logic fusion operators.

In this scenario, the final reliability measure calculated was $\phi = 0.018$ on a scale of 0 to 1. Such a low score would trigger an immediate visual alert within the EHR interface, signaling to subsequent clinicians that the diagnosis requires independent verification rather than being accepted at face value.

Conclusion and Future Work

The MDRA model demonstrates that metadata can serve as a dynamic safety signal rather than just a static index. By marking rather than deleting unreliable data, the system preserves potentially useful information while providing clinicians with the context needed to exercise appropriate caution.

Future research will focus on integrating this theoretical model into active clinical workflow engines, such as the YAWL system. Additionally, further work is required to develop "reputation recovery" mechanisms and to establish the systemic incentives necessary to ensure consistent feedback reporting from medical professionals.

Summary updated

Abstract

There is currently a strong focus worldwide on the potential of large-scale Electronic Health Record (EHR) systems to cut costs and improve patient outcomes through increased efficiency. This is accomplished by aggregating medical data from isolated Electronic Medical Record databases maintained by different healthcare providers. Concerns about the privacy and reliability of Electronic Health Records are crucial to healthcare service consumers. Traditional security mechanisms are designed to satisfy confidentiality, integrity, and availability requirements, but they fail to provide a measurement tool for data reliability from a data entry perspective. In this paper, we introduce a Medical Data Reliability Assessment (MDRA) service model to assess the reliability of medical data by evaluating the trustworthiness of its sources, usually the healthcare provider which created the data and the medical practitioner who diagnosed the patient and authorised entry of this data into the patient's medical record. The result is then expressed by manipulating health record metadata to alert medical practitioners relying on the information to possible reliability problems.
BibTeX
@article{alhaqbani2009medical,
  author = {Bandar Alhaqbani and Audun Jøsang and Colin Fidge},
  title = {{A Medical Data Reliability Assessment Model}},
  journal = {Journal of Theoretical and Applied Electronic Commerce Research},
  year = {2009},
  doi = {10.4067/s0718-18762009000200006},
}
Figure