Discovery of the Latent Supportive Relevance in Medical Data Mining

Sam Chao, Derek F. Wong, Qiulin Ding, Yiping Li, Mingchui DongInTech 2008 · DOI · OpenAlex

Abstract

In medical data mining, diagnostic accuracy is the most critical metric. Traditional data pre-processing methods, such as Feature Selection (FS) and Continuous Feature Discretization (CFD), often employ univariate approaches that evaluate attributes in isolation. This paper introduces a novel framework for improving classification accuracy by uncovering "latent supportive relevance"—hidden inter-correlations between attributes that are individually weak but collectively powerful. We propose two multivariate techniques: MIDCA for discretization and LUIFS for feature selection. Experimental results across 12 UCI datasets demonstrate that these methods significantly improve the performance of standard learning algorithms like ID3 and C4.5 by rescuing valuable information typically discarded during pre-processing.

1. Introduction

The effectiveness of machine learning in the medical domain is heavily dependent on the quality of data preparation. Two critical steps in this pipeline are discretization—converting continuous variables into categorical intervals—and feature selection—identifying the most relevant subset of variables for the model. However, most common pre-processing methods are univariate, meaning they ignore the interdependent relationships between features.

In clinical settings, a single symptom may be non-specific. For instance, a patient's age or weight might not be predictive of a specific condition on its own. However, when combined, they define a composite metric like Body Mass Index (BMI) or age-adjusted hypertension risk. If a pre-processing algorithm discards "age" because it lacks individual correlation with the target, the model loses the ability to form these crucial compound conditions. This paper addresses this gap by proposing methods to identify and retain features with "latent supportive relevance."

2. Proposed Methodology

The authors introduce two primary algorithms designed to capture feature interactions before the training phase.

2.1 MIDCA: Multivariate Interdependent Discretization

Standard discretization methods often use global bins or simple entropy-based splits on a single variable. MIDCA (Multivariate Interdependent Discretization for Continuous Attribute) instead seeks an "interdependent attribute" to guide the discretization process. By pairing a continuous variable with another relevant attribute (identified via normalized Information Gain and Relief measures), MIDCA determines cutting points that reflect the contextual relationship between the two, ensuring that the resulting discrete intervals are more diagnostically meaningful.

2.2 LUIFS: Latent Utility of Irrelevant Feature Selection

LUIFS is a two-phase filter designed to prevent the premature removal of features that possess latent utility.

  1. Attribute Sorting Phase: Features are ranked using Information Gain. Those falling below a calculated mean threshold are initially labeled "irrelevant."
  2. Latent Utility Searching Phase: Rather than discarding these irrelevant features, LUIFS evaluates them in pairs with other features using a combinatorial Relief algorithm. If an "irrelevant" feature significantly boosts the predictive power of another feature, it is "rescued" and included in the final optimal feature set.

3. Experimental Evaluation

The proposed methods were evaluated using 12 datasets from the UCI Machine Learning Repository, including medical datasets such as Cleve (heart disease), Hepatitis, Hypothyroid, and Diabetes. The performance was measured using the classification error rates of ID3 and C4.5 decision tree algorithms.

4. Results and Discussion

The experimental results confirm that capturing latent relevance leads to superior model performance.

  • Discretization Gains: Using MIDCA reduced the average classification error for the ID3 algorithm from 15.27% (using standard univariate discretization) to 13.36%. For the C4.5 algorithm, the error rate dropped from 16.18% to 13.49%, representing a relative improvement of approximately 16.6%.
  • Feature Selection Gains: LUIFS consistently outperformed both "No Feature Selection" (NoFS) and standard "Attribute Ranking Feature Selection" (ARFS). For ID3, LUIFS improved accuracy by approximately 11% over both benchmarks. In the case of C4.5, which includes internal pruning mechanisms, LUIFS still provided an 8.5% improvement over NoFS.
  • Feature Rescue: The LUIFS algorithm successfully identified and retained between 1 and 8 "latent" features per dataset—variables that standard univariate filters would have discarded but which proved essential for accurate classification.

5. Conclusion and Future Work

Uncovering latent supportive relevance is a key factor in maximizing the performance of learning algorithms in data-sensitive fields like medicine. By shifting from univariate to multivariate pre-processing, MIDCA and LUIFS preserve hidden valuable information that traditional methods overlook.

While these methods introduce higher computational complexity—$O(N^2)$ due to the pairwise evaluation of features—this is often a justifiable trade-off for the gains in diagnostic accuracy. Future research will focus on optimizing these algorithms for high-dimensional data and testing their efficacy with other learning models, such as Naive-Bayes and Artificial Neural Networks.

Summary updated

Abstract

5.1. Conclusions In this paper, we have proposed a novel concept of improving the classification accuracy for common learning algorithms by uncovering the potential attributes relevance through multivariate interdependent attribute pre-processing methods. However, many common pre-processing methods ignored the latent inter-correlation between attributes, this sometimes may loose the potential hidden valuable information, thus limit the performance of the learning algorithms. Especially in medical domain, diagnostic accuracy is treated as the most important issue, in order to approach to the highest accuracy, qualities of the preprocessing methods are quite essential, thereby finding out the most useful attributes relevance is a key factor for a successful method. Dataset Cleve Hepatitis Hypothyroid Heart Sick-euthyroid Auto Breast Diabetes Mushroom Parity5+5 Corral Led7 Average NoFS 20 15.6 1.1 17.8 2.5 27.1 5.7 30.9 0 50 9.4 32.6 17.72 ARFS 22.2 7.9 0.4 14.4 2.5 21.9 5.6 30.1 0 50 12.5 43.7 17.6 LUIFS 19.3 7.9 0.4 15.6 2.4 21.8 5.4 30.1 0 50 9.4 32.3 16.21
BibTeX
@inproceedings{chao2008discovery,
  author = {Sam Chao and Derek F. Wong and Qiulin Ding and Yiping Li and Mingchui Dong},
  title = {{Discovery of the Latent Supportive Relevance in Medical Data Mining}},
  booktitle = {InTech},
  year = {2008},
  doi = {10.5772/6404},
}
Figure