Smartphone-Based Colorimetric Analysis of Urine Test Strips for At-Home Prenatal Care

Madeleine Flaucher, Michael Nissen, Katharina M. Jaeger, Adriana Titzmann, Constanza Pontones, Hanna Huebner, Peter A. Fasching, Matthias W. Beckmann, Stefan Gradl, Bjoern M. EskofierIEEE Journal of Translational Engineering in Health and Medicine 2022 · DOI

TL;DR

This paper presents a smartphone-based pipeline for automated analysis of urine test strips, enabling at-home prenatal care. By leveraging image processing and machine learning, the system aims to reduce human error, increase accuracy, and shift medical examinations to the home environment.

What Problem Does This Paper Address?

Clinical urinalysis using test strips is a crucial part of prenatal care for early detection of various health issues. However, the current method relies on visual interpretation of color changes on the test strips against a reference card. This visual assessment is time-consuming, prone to operator-dependent errors, and often not performed consistently due to the high cost of automated analysis devices. Furthermore, the lack of proper documentation of results, especially negative ones, hinders historical tracking of a patient's health. The paper addresses the need for a more accessible, accurate, and convenient method for urinalysis, particularly for at-home use, to improve patient care and reduce the burden on medical staff.

Methods

The core of this work is an automated pipeline for urine test strip analysis using smartphone images. The pipeline involves several key stages:

  • Object Detection:
    • Reference Card Detection: A feature matching approach (specifically using ORB descriptors) was employed to robustly detect the reference card in images, achieving high accuracy.
  • Urine Stick Detection: Two methods were explored: feature matching (which proved insufficient) and a region-based convolutional neural network (R-CNN), specifically Mask R-CNN. The Mask R-CNN approach, leveraging transfer learning from a pre-trained model, demonstrated significantly better performance.
  • Reference Field Detection: Once the reference card is detected, its individual color fields are extracted using pre-measured pixel coordinates from a reference image.
  • Test Field Detection: Due to the variability of urine sticks, a custom pipeline involving edge detection and k-means clustering was developed to accurately locate the individual test fields. This method addresses challenges posed by different lighting conditions and perspectives.
  • Color Comparison: Three distinct methods were used to compare the colors of the test fields with their corresponding reference fields:
  • Hue Channel Comparison: Directly comparing the dominant wavelength of colors.
  • Matching Factor: A weighted comparison across HSV color channels.
  • Euclidean Distance: Calculating the distance between color points in the HSV color space.
  • Test Result Determination: The color comparison method yielding the highest similarity for each parameter determines the final test result. These results are then compared against visually determined ground truth.

Datasets

The study utilized two datasets:

  • At-Home Study Dataset: Consisting of 150 images captured by 26 participants in their home environments using their own smartphones. These images were visually evaluated and supplemented with questionnaires.
  • Control Urine Study Dataset: Comprising 135 images from a controlled laboratory setting using manufactured control urine. This dataset was used to simulate healthy and specific pathological ranges.
  • Supplementary Images: An additional 74 images were captured to improve the development of the object detection pipeline, featuring challenging conditions like unusual backgrounds, lighting, and perspectives.

Key Insights and Ideas

  • Feasibility of At-Home Urinalysis: The study strongly suggests that automated urine test strip analysis is feasible in a home environment using readily available smartphones.
  • Mask R-CNN for Urine Stick Detection: The Mask R-CNN approach significantly outperforms traditional feature matching for detecting urine test strips, highlighting the power of deep learning for this task.
  • Multi-Method Color Comparison: No single color comparison method (Hue, Matching Factor, Euclidean Distance) was universally superior for all parameters. This suggests that a hybrid approach or parameter-specific optimization might be necessary for optimal accuracy.
  • User Experience Matters: The at-home study revealed that participants often struggled with adhering to the precise timing for test strip evaluation, indicating a need for user-friendly interfaces and potentially more forgiving test strip designs.
  • Potential for Digital Health Integration: The work lays the groundwork for integrating such smartphone-based diagnostics into broader digital health platforms for remote patient monitoring and care.

Key Results

  • Object Detection Accuracy:
    • Reference Card Detection: 98.6% accuracy.
  • Urine Stick Detection (Mask R-CNN): 85.5% accuracy.
  • Color Comparison Performance:
    • Average F1-Scores: Hue channel (0.81), Matching Factor (0.80), Euclidean distance (0.70).
  • All methods achieved perfect classification for pH values.
  • Leukocyte results showed a high rate of false positives across all methods.
  • Usability: The average System Usability Scale (SUS) score was 71.9 ± 17.3, indicating a generally good but improvable user experience.
  • Time Adherence: A significant portion of participants did not adhere to the manufacturer's specified time intervals for test strip evaluation.

Limitations and Future Work

  • Limitations:
    • Limited Range of Measured Values: The study primarily used healthy participants and control urine with only two specific levels, limiting the coverage of pathological values.
  • Ground Truth Accuracy: The visual evaluation used as ground truth may be subject to errors, especially given the timing issues encountered by participants.
  • Small Sample Size for Certain Analyses: The number of participants and the variety of smartphones used were limited, preventing detailed statistical analysis of device influence or educational background impact.
  • Future Work:
    • Improved Ground Truth: Incorporating a more accurate color analysis system for ground truth validation.
  • Enhanced Usability: Improving the web application and potentially exploring easier-to-use test strips to address timing and user confusion.
  • Larger Datasets and Studies: Conducting larger studies with a wider range of measurement values and in real-world antenatal care settings.
  • Real-time Feedback: Developing the capability to provide results to users on their smartphones in real-time.
  • Parameter Tuning: Further optimizing detection and color comparison parameters, potentially exploring combinations of methods.
  • Investigating Sensing Parameters: Analyzing the influence of smartphone camera sensing parameters on accuracy.

Summary updated

Abstract

The findings demonstrate that routine urine examinations can be transferred into the home environment using a smartphone. Simultaneously, human error is avoided, accuracy is increased and medical staff is relieved.
BibTeX
@article{flaucher2022smartphone,
  author = {Madeleine Flaucher and Michael Nissen and Katharina M. Jaeger and Adriana Titzmann and Constanza Pontones and Hanna Huebner and Peter A. Fasching and Matthias W. Beckmann and Stefan Gradl and Bjoern M. Eskofier},
  title = {{Smartphone-Based Colorimetric Analysis of Urine Test Strips for At-Home Prenatal Care}},
  journal = {IEEE Journal of Translational Engineering in Health and Medicine},
  year = {2022},
  doi = {10.1109/jtehm.2022.3179147},
}
Figure