Toward medical test recommendation from optimal attribute selection perspectives: a backward reasoning approach

Nengjun Zhu, Jieyun Huang, Jian Cao, Liang Hu, Siji ZhuComplex 2024 · DOI

Abstract

Medical tests are crucial for treatment decision-making, yet over-testing imposes significant financial burdens and wastes clinical resources. Identifying which medical test items (MTIs) are necessary for a specific patient is a non-trivial task due to diverse health statuses and complex therapy prerequisites. This paper proposes a data-driven approach to prioritize MTIs by modeling the relationships between tests and therapies. We introduce the Dual Hierarchical Topic Model (DHTM), which treats hierarchical therapies as labeled topics and MTI reports as documents. By leveraging therapy-attribute distributions, we scope candidate therapies to evaluate the accumulated gain of potential tests. Extensive experiments on real-world medical data demonstrate that our approach effectively recommends the most informative tests, significantly outperforming traditional baselines.

Introduction

In modern clinical practice, the proliferation of medical test items (MTIs)—ranging from routine blood work to advanced genomic sequencing—has led to a growing challenge of "over-testing." This phenomenon not only strains healthcare budgets but also delays critical treatment. The core difficulty in optimizing test selection lies in the nature of medical data: it is often highly sparse, deeply personalized, and inherently hierarchical.

To address this, we introduce MtiRec, a recommendation framework designed to identify the most informative tests while minimizing redundancy. MtiRec "reasons backward" from potential treatments to determine which missing test results would provide the most decisive information for a final therapy decision.

Methodology

Dual Hierarchical Topic Model (DHTM)

The foundation of MtiRec is the Dual Hierarchical Topic Model (DHTM). Inspired by probabilistic latent semantic analysis in Natural Language Processing, DHTM treats a patient’s set of test results—represented as Attribute-Value Pairs (AVPs)—as a "document." Potential therapies are treated as "topics" or labels.

The Dual Hierarchical Topic Model (DHTM) architecture, illustrating the probabilistic dependencies between therapies and attribute-value pairs.

Unlike standard flat models, DHTM explicitly incorporates the hierarchical structure of both medical tests and therapies. For instance, it recognizes that a specific drug belongs to a broader class of therapy. This hierarchical awareness allows the model to make robust inferences even when patient data is extremely sparse, avoiding the noise introduced by handcrafted data padding.

Backward Reasoning and Therapy-Centric Logic

Most medical AI systems focus primarily on diagnosis. MtiRec shifts the focus to therapy selection, arguing that the primary utility of a medical test is its ability to narrow down the treatment path.

Conceptual framework of MtiRec illustrating how the model evaluates the 'gain' of unperformed tests based on their ability to confirm candidate therapies.

The model identifies "decisive AVPs"—test results that strongly differentiate between candidate therapies. By calculating the "accumulated gain" for each unperformed test, the system prioritizes MTIs that are most likely to lead to a definitive treatment plan. Furthermore, the model utilizes "Therapy Tuples" to capture the mutual dependencies between concurrent treatments, such as the combination of specific chemotherapy agents with radiotherapy.

Experimental Evaluation

The researchers evaluated MtiRec using two distinct real-world datasets:

  1. BCDB (Breast Cancer Database): A sparse dataset of 7,040 patients with 132 attributes (68.14% missing data).
  2. MMDB (Postnatal Depression Dataset): A denser dataset of 1,189 patients with 78 attributes (15.64% missing data).

Key Results

MtiRec significantly outperformed standard baselines, including AdaBoost, Logistic Regression, and deep learning architectures like DPCNN.

  • Accuracy and Precision: On the BCDB dataset, MtiRec improved Accuracy@5 (the probability that the correct missing test is in the top 5 recommendations) by 12.7% compared to its non-hierarchical variant.
  • Prioritizing Impact over Popularity: While simple statistical models often recommend common tests (e.g., height or weight), MtiRec successfully identifies "low-frequency but high-impact" tests that are critical for specific therapy decisions.
  • Mean Reciprocal Rank (MRR): The model consistently placed the most decisive tests at the top of the recommendation list, optimizing clinical workflow and saving clinician time.

Conclusion and Future Work

MtiRec demonstrates that modeling the structural hierarchy of medical knowledge is more effective for test recommendation than simply increasing data volume. While the current probabilistic graphical model is highly effective, future work will explore integrating deep learning to handle non-linear relationships and developing "test packages" to align with standard clinical panels. Additionally, establishing dynamic thresholds for "stopping points" remains a key area for future research.

Summary updated

Abstract

Medical tests are crucial for treatment decision making. However, over-testing can often occur in any medical speciality or level of expertise. Since over-testing usually results in a financial burden for patients and is also a waste of medical resources, this naturally leads to the question: which medical test items (MTIs) are necessary and should be prioritized for the target patients? It is a nontrivial task to identify the right MTIs due to the diversified health status of patients and the complicated prerequisites of therapies. To this end, in this paper, we propose a data-driven approach to evaluate the priority which should be given to MTIs by modeling the relationships between MTIs and therapies. Specifically, we first develop a dual hierarchical topic model (DHTM), which views the adopted hierarchical therapies as labeled topics and the MTI reports, i.e., the set of hierarchical attribute-value pairs (AVPs), as documents. Then, with the therapy-AVP distribution and the partial MTI reports of the target patient, we can scope the candidate therapies, which are further utilized to evaluate the accumulated gain of MTIs to be tested. Moreover, the next MTI recommendation is conducted based on the gains. Finally, extensive experiments on real-world medical data validate the effectiveness of our approach, and some interesting observations are also provided.
BibTeX
@inproceedings{zhu2024toward,
  author = {Nengjun Zhu and Jieyun Huang and Jian Cao and Liang Hu and Siji Zhu},
  title = {{Toward medical test recommendation from optimal attribute selection perspectives: a backward reasoning approach}},
  booktitle = {Complex},
  year = {2024},
  doi = {10.1007/s40747-024-01629-3},
}
Figure