A study of the moving rate of positive results for use in a patient‐based real‐time quality control program on a procalcitonin point‐of‐care testing analyzer

Yili He, Daqing Gu, Xiangzhi Kong, Zhiqiang Feng, Weishang Lin, Yunfeng CaiJournal of Clinical Laboratory Analysis 2022 · DOI

TL;DR

This study introduces a novel "moving rate of positive results" (MR) quality control program for procalcitonin point-of-care testing (POCT). The MR program demonstrates superior sensitivity in detecting small analytical biases compared to traditional methods and the moving average (MA) approach, making it a promising tool for ensuring the accuracy of POCT results in clinical settings.

What Problem Does This Paper Address?

Point-of-care testing (POCT) offers rapid results but faces challenges in maintaining analytical accuracy. Traditional quality control (QC) methods, often relying on external QC products, can be insufficient for detecting subtle analytical errors that might impact patient outcomes. These external controls can also be affected by matrix effects. While patient-based real-time quality control (PBRTQC) methods, like the moving average (MA), have been proposed, they often struggle to detect small biases quickly. This paper addresses the need for a highly sensitive and efficient PBRTQC program that can reliably monitor the quality of POCT results, specifically for procalcitonin (PCT), a crucial biomarker for diagnosing bacterial infections.

Methods

The researchers developed and evaluated two PBRTQC programs: a Moving Average (MA) program and a Moving Rate of Positive Results (MR) program.

  • MA Program: This program calculates a rolling average of recent patient results. It incorporates a truncation limit (TL) to exclude extreme values and a block size (N) for averaging. Control limits (CLs) are set based on the mean and standard deviation of the MA.
  • MR Program: This innovative program converts patient results into binary values (0 or 1) based on a cut-off value (COV). It then calculates the "moving rate of positive results" within a rolling window (block size N). Control limits (CLs) are derived from the mean and standard deviation of this rate.

To assess their performance, the researchers used a Monte Carlo simulation to introduce artificial positive and negative biases into a dataset of 2434 procalcitonin POCT results. They then evaluated the programs based on:

  • Median number of patient samples affected until error detection (MNPed): A lower MNPed indicates faster error detection.
  • False Rejection Rate (FR): The rate of incorrectly flagging a good run as out-of-control.
  • Type II error (β): The rate of failing to detect a true bias.

Optimal parameters for both MA (truncation limit 'i' and block size N) and MR (cut-off value COV and block size N) were determined through simulation, aiming for low FR and β values while minimizing MNPed.

Datasets

The study utilized a retrospective dataset of 2434 procalcitonin (PCT) patient samples tested at the Fifth People's Hospital of Panyu District, Guangzhou, from July 2019 to June 2020. This dataset was used for both training (first 1000 samples) and testing (remaining 1434 samples) of the PBRTQC programs. No new benchmark datasets were introduced; rather, existing patient data was used to simulate biases.

Key Insights and Ideas

  • The MR program's unique approach: By converting results to a binary "positive/negative" state based on a COV, the MR program bypasses the need for complex data smoothing (like TL in MA) and directly quantifies the rate of positive results, offering a simpler yet effective way to monitor trends.
  • Sensitivity to small biases: The MR program's core innovation lies in its ability to detect very small analytical shifts that traditional QC and even the MA program miss. This is crucial because even minor biases can accumulate and lead to incorrect clinical interpretations.
  • Adaptability to skewed data: Procalcitonin data often exhibits a skewed distribution. The MR program's binary conversion and rate calculation appear more robust to such distributions compared to the MA program, which relies on mean and standard deviation calculations that can be heavily influenced by skewness.
  • Potential for dual parameter optimization: The MR program's ability to use different COVs for positive and negative biases suggests a potential for tailored sensitivity in different error directions, a flexibility not explicitly explored in the MA program.

Key Results

  • Conventional QC and MA limitations: The study found that conventional QC and the MA program were not sensitive enough to detect small biases (e.g., 0.01-0.1 ng/ml). Conventional QC, specifically the 4₁s rule, required a bias of at least 0.24 ng/ml to be reliably detected.
  • MR program's superior detection: The MR program, with optimized parameters (COV = 0.03 or 0.07, N = 10), demonstrated the ability to detect:
  • Positive biases as low as 0.06 ng/ml with a β < 0.33%.
  • Negative biases as low as 0.4 ng/ml with a β < 0.50%.
  • Low false rejection: The MR program achieved low false rejection rates (FRs < 1.0%) and type II errors (β < 1%) with its optimal parameters.
  • Efficiency: The MR program is described as having higher data utilization and simpler calculation procedures, making it suitable for laboratories with smaller daily data volumes, a common scenario in POCT.

Limitations and Future Work

  • Limitations:
    • The study acknowledges that it could not exclude the possibility of not detecting all systematic errors, especially if they occurred during instrument maintenance or calibration periods.
  • The biases introduced in the simulation were simplified (e.g., continuous positive/negative biases). Real-world experimental biases might be more complex and variable.
  • The study focused on PCT, and the applicability of the MR program to other analytes with different distributions would require further validation.
  • The use of dual COVs for the MR program, while promising, requires further experimental validation to confirm its effectiveness across various data models.
  • Future Work:
    • Further experimental validation of the MR program with dual COVs for different data models is suggested.
  • The authors imply that the MR program's adaptability makes it suitable for a wide range of POCT scenarios, including those with low specimen volumes, suggesting its broader application.
  • The paper highlights the need for more sensitive and intelligent QC programs to address current challenges in POCT operator practices, implying that the MR program could be a step towards this goal.

Summary updated

Abstract

The MR program, which is more sensitive to small biases than conventional QC and MA programs, with low FR and β. As such, it can be used as a PBRTQC program with high performance.
BibTeX
@article{he2022study,
  author = {Yili He and Daqing Gu and Xiangzhi Kong and Zhiqiang Feng and Weishang Lin and Yunfeng Cai},
  title = {{A study of the moving rate of positive results for use in a patient‐based real‐time quality control program on a procalcitonin point‐of‐care testing analyzer}},
  journal = {Journal of Clinical Laboratory Analysis},
  year = {2022},
  doi = {10.1002/jcla.24320},
}

Loading related content...

Figure