Abstract
Adaptive Resonance Theory (ART) is considered an effective approach for realizing continual learning thanks to its ability to handle the plasticity-stability dilemma. In general, however, the clustering performance of ART-based algorithms strongly depends on the specification of a similarity threshold, i.e., a vigilance parameter, which is data-dependent and specified by hand. This paper proposes an ART-based topological clustering algorithm with a mechanism that automatically estimates a similarity threshold from the distribution of data points. In addition, for improving information extraction performance, a divisive hierarchical clustering algorithm capable of continual learning is proposed by introducing a hierarchical structure to the proposed algorithm. Experimental results demonstrate that the proposed algorithm has high clustering performance comparable with recently-proposed state-of-the-art hierarchical clustering algorithms.
Introduction: The Plasticity-Stability Dilemma
In the field of machine learning, models often struggle with the plasticity-stability dilemma. This refers to the challenge of remaining "plastic" enough to learn new patterns from a continuous stream of data while remaining "stable" enough to prevent the overwriting of previously acquired knowledge—a phenomenon known as catastrophic forgetting.
Adaptive Resonance Theory (ART) was specifically designed to address this dilemma. ART-based models only create new clusters when input data is sufficiently different from existing knowledge. However, ART has a significant practical drawback: the vigilance parameter. This is a manually set similarity threshold that determines when a new cluster should be formed. If set too high, the model over-segments the data; if set too low, it merges distinct groups into a single, meaningless cluster. Furthermore, most ART models provide a "flat" view of data, failing to capture the hierarchical relationships (e.g., species within a genus) that are common in real-world datasets.
Methodology: CAEA and HCAEA
To solve these issues, the authors propose a two-tiered framework consisting of a base clustering algorithm and a hierarchical wrapper.
1. CAEA (CIM-based ART with Edge and Age)
CAEA is the core clustering engine. It introduces two major improvements over traditional ART:
- Correntropy-Induced Metric (CIM): Instead of using standard Euclidean distance, which is highly sensitive to outliers, CAEA uses CIM. This local metric effectively ignores "noise" data points that fall outside the main distribution, leading to more robust cluster centers.
- Automated Thresholding: CAEA eliminates the need for manual vigilance tuning. It uses a Kernel Density Estimator (KDE) to analyze the distribution of the first few data points. By automatically calculating the bandwidth (σ) of the data density, the algorithm sets its own sensitivity threshold, making it a "plug-and-play" solution for different datasets.
2. HCAEA (Hierarchical CAEA)
HCAEA extends CAEA into a divisive hierarchical structure. The process begins with a single CAEA model at the root level. Once this model is trained on the initial data, the points assigned to each resulting cluster are passed down to "child" CAEA models in the next layer. This top-down approach allows the system to discover fine-grained sub-clusters within broader categories. Because it utilizes ART logic at every level, HCAEA can adaptively generate new partitions or branches as new data classes appear, making it inherently capable of continual learning.
Experimental Evaluation
The researchers evaluated HCAEA using a comprehensive suite of benchmarks:
- Synthetic Datasets: Five datasets (Aggregation, Compound, Hard Distribution, Jain, and Pathbased) were used to test the model's ability to handle complex, non-spherical geometric shapes.
- Real-World Datasets: Nine datasets from the UCI repository were tested, ranging from low-dimensional sets like Iris and Wine to high-dimensional data such as COIL20 (1,024 attributes) and TOX171 (5,748 attributes).
Performance was compared against state-of-the-art hierarchical models, including GH-EXIN and HFTCA, in both stationary (batch) and non-stationary (sequential) environments.
Key Results and Insights
The experimental results highlighted several strengths of the HCAEA approach:
- High Accuracy: In both stationary and non-stationary tests, HCAEA consistently ranked among the top performers. Statistical analysis (Friedman and Nemenyi tests) confirmed that HCAEA and GH-EXIN were the superior models overall across the entire benchmark suite.
- Stability in Continual Learning: In non-stationary tests where classes were introduced one by one, HCAEA maintained high accuracy on early classes even after learning later ones. This proves its effectiveness in mitigating catastrophic forgetting.
- Structural Efficiency: While competing models like GH-EXIN often generated a massive number of nodes (sometimes in the thousands) to achieve accuracy, HCAEA maintained a much more compact and interpretable hierarchy.
- Robustness: The use of CIM allowed the model to maintain performance even in the presence of significant noise and outliers that typically degrade standard K-means or ART variants.
Conclusion and Future Work
HCAEA represents a significant advancement in automated clustering. By combining the stability of ART with automated threshold estimation and a divisive hierarchy, the authors have created a system that can grow its own structure to match data complexity without human intervention.
Future research will focus on addressing "node explosion" in specific high-density datasets by introducing pruning mechanisms. Additionally, the authors aim to improve the model's resilience to initial sample bias and explore its performance in environments with "concept drift," where the definition of existing classes changes over time.
Summary updated