An Evolutionary-Based Approach to Learning Multiple Decision Models from Underrepresented Data

Vitaly Schetinin, Dayou Li, Carsten Maple

Introduction

In high-stakes fields like medical diagnostics, researchers often face the challenge of "underrepresented" data—scenarios where only a handful of verified patient cases are available for training. Standard machine learning approaches, particularly deep neural networks, struggle in these environments due to two primary issues: overfitting and a lack of interpretability. When data is scarce, complex models tend to memorize specific noise rather than generalizable patterns. Furthermore, clinical settings require transparent "if-then" logic so that practitioners can understand the reasoning behind a diagnosis, such as differentiating between Infectious Endocarditis and Systemic Red Lupus.

This paper introduces the Evolving Decision Model (EDM) framework, designed to maximize performance and transparency from small datasets. By evolving ensembles of simple logical rules and utilizing leave-one-out cross-validation (LOO-CV), the EDM approach provides a robust alternative to traditional "black-box" models.

The EDM Framework

The core of the EDM approach is the construction of an ensemble of Decision Models (DMs) through an evolutionary process. Rather than mapping complex inputs to outputs in a single step, the system utilizes Logical Decomposition. Complex decision functions are broken down into a series of simple units, each processing only two arguments using one of ten basic logical functions.

Basic logical functions used for model decomposition.

The models "grow" iteratively, adding processing units only if they improve the model's performance according to a LOO-CV score. For numeric data, the system employs Entropy-Based Thresholding to find optimal cut-off points that minimize decision uncertainty. To ensure robustness, the system generates thousands of diverse models, forming an ensemble where the final diagnosis is determined by majority voting. This ensemble approach also allows for the calculation of a confidence score based on the level of agreement among individual models.

Experimental Setup

The EDM technique was evaluated across four diverse clinical and benchmark datasets:

  1. IE/SRL Data: 36 cases with 24 features for differentiating Infectious Endocarditis from Systemic Red Lupus.
  2. UK Trauma Data: Real-world outcomes from trauma cases.
  3. SPECT Heart: Cardiac imaging data from the UCI repository.
  4. Wisconsin Prognostic Breast Cancer (WPBC): Cancer prognosis data.

To simulate the difficulties of underrepresented data, training was often restricted to small subsets (e.g., 50 samples), testing the models' ability to generalize from minimal information.

Results and Discussion

The EDM framework was benchmarked against a standard Artificial Neural Network (ANN) ensemble of 5,000 networks. The results consistently favored the evolutionary approach:

  • Accuracy: EDM outperformed the ANN ensemble in nearly every test. For instance, on the Trauma dataset, EDM reached 83.2% accuracy, surpassing the ANN's 80.8%.
  • Feature Selection: A significant advantage of the EDM is its inherent ability to identify critical diagnostic attributes. In the IE/SRL problem, the system successfully narrowed 24 clinical tests down to the 8 most significant features.
Ranking of clinical features based on their selection frequency in the ensemble.
  • Clinical Utility: The resulting models are directly convertible into human-readable rules, allowing clinicians to audit the logic. In real-world validation involving over 200 patients, the diagnostic tables produced by EDM maintained a misdiagnosis rate of less than 2%.

Conclusion

The EDM framework demonstrates that simplicity can be a powerful defense against overfitting in small-data regimes. By restricting individual unit complexity and leveraging the statistical rigor of LOO-CV, the system builds ensembles that are both accurate and interpretable. While the heuristic nature of the search does not guarantee a global optimum, the practical results in clinical diagnostics are highly promising. Future work may explore scaling these combinatorial searches to handle even larger sets of attributes more efficiently.

Summary updated

BibTeX
@inproceedings{schetinin2008evolutionary,
  author = {Vitaly Schetinin and Dayou Li and Carsten Maple},
  title = {{An Evolutionary-Based Approach to Learning Multiple Decision Models from Underrepresented Data}},
  booktitle = {ICNC},
  year = {2008},
  doi = {10.1109/icnc.2008.409},
}
Figure