Evaluation Model of Telemedicine Service Quality Based on Machine Sensing Vision

Yingdong Cao, Hui Li, Zeqi Xie, Zhenti Cui, Loknath Sai AmbatiEAI 2022 · DOI

TL;DR

This paper introduces a novel evaluation model for telemedicine service quality that leverages Machine Sensing Vision and Dynamic Vision Sensors (DVS). By combining event-based imaging with a hybrid "subjective-objective" weighting system, the researchers created a framework that significantly outperforms traditional survey-based methods in both accuracy and user satisfaction.

What Problem Does This Paper Address?

Telemedicine has bridged the gap for patients in remote areas, but measuring its "quality" remains a moving target. Traditional evaluation methods often rely on static surveys or narrow technical metrics (like bitrates), which fail to capture the nuances of doctor-patient interaction or the reliability of the system under real-world conditions.

The authors identify a "low accuracy, low satisfaction" loop: if you can't identify the key indicators that actually matter to users, you can't improve the service. This paper seeks to automate the identification of these indicators using machine vision and provide a mathematically rigorous way to weigh their importance.

Methods

The researchers built their model using a multi-layered approach:

  1. Information Acquisition (DVS): Instead of standard cameras, they use Dynamic Vision Sensors. Unlike traditional frame-based cameras, DVS only records changes in pixels (events). This allows for microsecond-level response times, lower data overhead, and better performance in high-speed or low-light scenarios.
  2. Image Processing: They employ Gaussian Mixture Models (GMM) for background subtraction and Mathematical Morphology (corrosion, expansion, opening, and closing operations) to clean the visual data and extract "bone structures" of the target objects.
  3. The Index System: They established five primary dimensions for quality: Network, System, Structural, Interaction, and Result quality.
  4. Hybrid Weighting: To determine which factors are most important, they combined:
  5. Subjective Weighting: Pairwise comparison based on expert opinions.
  6. Objective Weighting: The Entropy Method, which uses the actual dispersion of data to determine an indicator’s significance.
  7. Matter-Element Analysis: Finally, they used matter-element analysis and extension sets to transform qualitative service levels (e.g., "Excellent" vs. "Average") into quantitative scores.

Datasets

The study did not use a standard public benchmark. Instead, the authors generated their own data through:

  • Expert Questionnaires: Distributed to doctors in grass-roots and central hospitals, as well as telemedicine researchers, using a 7-point language evaluation scale.
  • Field Testing: The model was evaluated using real-world data from two specific medical facilities, referred to as the "First hospital" and "Second hospital."

Key Insights and Ideas

  • Event-Based Sensing for Healthcare: The use of DVS is a standout choice. By focusing on "streams of events" rather than frames, the system reduces the cost of data transmission—a critical factor for telemedicine in rural areas with poor internet.
  • Reducing Human Bias: By using the Entropy Method alongside expert opinions, the model prevents "subjective drift." If experts think an indicator is important, but the data shows it doesn't vary much across sessions, the model automatically adjusts its weight.
  • The "Matter-Element" Framework: Treating service quality as a "matter-element" allows the researchers to handle the inherent fuzziness of human language. It provides a bridge between "I feel this service was good" and a hard numerical score.

Key Results

The proposed model was compared against three established telemedicine evaluation methods (referred to in the paper as References [5], [6], and [7]).

  • Reliability: The Cronbach’s alpha (a measure of internal consistency) was consistently above 0.7, indicating high reliability.
  • Accuracy: The Barthel Index, used here to measure the accuracy of the quality assessment, was higher than 90 for the proposed method, while competing methods hovered between 60 and 80.
  • User Satisfaction: In tests across five different user groups, the proposed model achieved a satisfaction rate of over 90%, significantly higher than the baseline methods.

Limitations and Future Work

While the paper demonstrates strong quantitative gains, a few areas remain open for exploration:

  • Hardware Accessibility: DVS hardware is currently more specialized and expensive than standard webcams. The paper does not discuss the cost-benefit ratio of deploying such sensors at scale in rural clinics.
  • Contextual Specifics: The paper is somewhat abstract regarding what the machine vision is specifically tracking during a session (e.g., is it tracking the doctor's gestures, the clarity of a wound scan, or the patient's physical reactions?).
  • Future Directions: The authors suggest this model lays the foundation for more automated "decision-making basis" in telemedicine management, potentially leading to real-time quality alerts during a consultation.

Summary updated

Abstract

INTRODUCTION: At present, the common telemedicine service quality evaluation methods can not obtain the key evaluation indicators, which leads to the low accuracy and low user satisfaction. OBJECTIVES: This paper constructs a telemedicine service quality evaluation model based on machine vision technology. METHODS: Machine vision technology is used to obtain telemedicine service information, preliminarily select service quality assessment indicators, complete the selection of indicators, build a telemedicine service quality assessment indicator system, adopt subjective and objective combination method to calculate the weight of service quality assessment indicators, and combine matter element analysis method to build a telemedicine service quality assessment model. RESULTS: The experimental results show that the Cronhach a is higher than 0.7, the Barthel index is higher than 90, and the satisfaction of many users is more than 90%. CONCLUSION: The proposed method solves the problems existing in the current method and lays a foundation for the development of telemedicine service technology.
BibTeX
@inproceedings{cao2022evaluation,
  author = {Yingdong Cao and Hui Li and Zeqi Xie and Zhenti Cui and Loknath Sai Ambati},
  title = {{Evaluation Model of Telemedicine Service Quality Based on Machine Sensing Vision}},
  booktitle = {EAI},
  year = {2022},
  doi = {10.4108/eetpht.v8i3.669},
}
Figure