An Ensemble Semi-Supervised Adaptive Resonance Theory Model With Explanation Capability for Pattern Classification

Farhad Pourpanah, Chee Peng Lim, Ali Etemad, Q. M. Jonathan Wu

TL;DR

This paper introduces SSL-ART and its ensemble version WESSL-ART, a new framework for semi-supervised learning based on Adaptive Resonance Theory. Unlike traditional "black-box" models, this approach learns incrementally in a single pass, handles unlabeled data to improve accuracy, and provides human-readable "If-Then" rules to explain its classifications.

What Problem Does This Paper Address?

Semi-supervised learning (SSL) is vital when labeled data is scarce but unlabeled data is plentiful (e.g., medical imaging or fault diagnosis). However, most existing SSL models suffer from three major drawbacks:

  1. Complexity & Iteration: They often require complex architectures and multiple passes over the data to converge.
  2. The "Black Box" Problem: It is difficult for users to understand why a model made a specific prediction.
  3. Catastrophic Forgetting: Many neural networks struggle with incremental learning, where new information often overwrites previously learned knowledge.

The authors aim to create a model that is efficient (one-pass learning), stable (incremental), and transparent (explainable).

Methods

The authors build upon Adaptive Resonance Theory (ART), a family of networks known for solving the "stability-plasticity dilemma." Their contribution consists of two main components:

  • SSL-ART (The Base Model):
    • Stage 1 (Unsupervised): Uses a Fuzzy ART network to cluster unlabeled samples into "prototype nodes."
    • Stage 2 (Supervised): Uses a Fuzzy ARTMAP structure to link these prototypes to specific class labels using a small set of labeled data.
  • One-to-Many (OtM) Mapping: A novel scheme where a single prototype can be associated with multiple classes during training. The model eventually assigns the prototype to the class it encountered most frequently. This reduces redundant nodes and makes the model more robust to noisy data.
  • WESSL-ART (The Ensemble): Because ART models are sensitive to the order in which they see data, the authors created an ensemble. Each member is trained on a different data sequence, and a weighted voting strategy is used to combine their predictions based on their class-specific performance on a validation set.
  • Rule Extraction: The model can translate its internal "prototype nodes" into fuzzy linguistic rules (e.g., "If Age is Very Small AND Heart Rate is Medium, then Class is Healthy"), providing a clear explanation facility.

Datasets

The researchers evaluated their models on a massive battery of tests:

  • 18 UCI Benchmarks: Standard datasets like Iris, Wine, MNIST, and Pima Indians Diabetes.
  • 3 Anomaly Detection Datasets: Tennessee Eastman process (TEP), Water Distribution (WADI), and UNSW-NB15 (network traffic).
  • Real-World Case Study: A human motion detection dataset using smartphone accelerometer data to distinguish between walking and running.

Key Insights and Ideas

  • Efficiency through One-Pass Learning: Unlike many SSL methods that require iterative labeling of the entire dataset, SSL-ART processes samples one by one. This makes it highly suitable for real-time, streaming data applications.
  • The Power of OtM Mapping: Standard ARTMAP uses one-to-one mapping, which can lead to "category proliferation" (creating too many nodes) when data is noisy. The OtM scheme acts as a natural filter, simplifying the network structure without sacrificing accuracy.
  • Transparency is a Feature, Not an Afterthought: By quantizing input features into linguistic terms, the model bridges the gap between high-dimensional math and human reasoning, which is critical for safety-related fields like medicine or industrial monitoring.

Key Results

  • Superior Accuracy: WESSL-ART consistently outperformed other SSL methods like SemiBoost and RegBoost across most UCI datasets.
  • Anomaly Detection: In the TEP and WADI datasets, WESSL-ART achieved F1-scores of 0.9378 and 0.9572 respectively, outperforming state-of-the-art models like GANF and HiSTAR.
  • Robustness to Noise: In the human motion study, the OtM mapping scheme proved significantly more resilient to label noise (incorrectly labeled training data) than the standard one-to-one mapping.
  • Model Compression: The rule extraction process produced a small, manageable set of rules—often three times fewer than the number of unlabeled training samples—making the "logic" of the model easy to audit.

Limitations and Future Work

The authors acknowledge that while the One-to-Many (OtM) mapping simplifies the model, it can occasionally lead to a loss of fine-grained information at the decision boundaries between classes.

Future directions include: * Refining the OtM mapping to better preserve boundary details. * Integrating reinforcement learning or probability-based methods to further enhance the model's robustness in highly uncertain environments.

Summary updated

Abstract

Most semi-supervised learning (SSL) models entail complex structures and iterative training processes as well as face difficulties in interpreting their predictions to users. To address these issues, this article proposes a new interpretable SSL model using the supervised and unsupervised Adaptive Resonance Theory (ART) family of networks, which is denoted as SSL-ART. Firstly, SSL-ART adopts an unsupervised fuzzy ART network to create a number of prototype nodes using unlabeled samples. Then, it leverages a supervised fuzzy ARTMAP structure to map the established prototype nodes to the target classes using labeled samples. Specifically, a one-to-many (OtM) mapping scheme is devised to associate a prototype node with more than one class label. The main advantages of SSL-ART include the capability of: (i) performing online learning, (ii) reducing the number of redundant prototype nodes through the OtM mapping scheme and minimizing the effects of noisy samples, and (iii) providing an explanation facility for users to interpret the predicted outcomes. In addition, a weighted voting strategy is introduced to form an ensemble SSL-ART model, which is denoted as WESSL-ART. Every ensemble member, i.e., SSL-ART, assigns a different weight to each class based on its performance pertaining to the corresponding class. The aim is to mitigate the effects of training data sequences on all SSL-ART members and improve the overall performance of WESSL-ART. The experimental results on eighteen benchmark data sets, three artificially generated data sets, and a real-world case study indicate the benefits of the proposed SSL-ART and WESSL-ART models for tackling pattern classification problems.
BibTeX
@article{pourpanah2024ensemble,
  author = {Farhad Pourpanah and Chee Peng Lim and Ali Etemad and Q. M. Jonathan Wu},
  title = {{An Ensemble Semi-Supervised Adaptive Resonance Theory Model With Explanation Capability for Pattern Classification}},
  journal = {IEEE Trans. Emerg. Top. Comput. Intell.},
  year = {2024},
  doi = {10.1109/tetci.2023.3285932},
}

Loading related content...

Figure