TL;DR
This paper introduces HCAEA, a new hierarchical clustering algorithm that solves two major pain points in machine learning: the need for manual parameter tuning and the "catastrophic forgetting" that happens during continual learning. By combining Adaptive Resonance Theory (ART) with automated threshold estimation, the authors created a system that grows its own structure to match data complexity without losing what it previously learned.
What Problem Does This Paper Address?
Most clustering algorithms struggle with the plasticity-stability dilemma: how do you stay "plastic" enough to learn new patterns without overwriting the "stable" knowledge you already have? While Adaptive Resonance Theory (ART) was designed specifically for this, it has a notorious Achilles' heel: the vigilance parameter. This is a manually set similarity threshold that determines when to create a new cluster. If you set it too high, you get too many clusters; too low, and your clusters become a meaningless blur.
Furthermore, standard clustering often provides a "flat" view of data. In the real world, data is often hierarchical (e.g., "Golden Retriever" $\subset$ "Dog" $\subset$ "Mammal"). Existing hierarchical models like Growing Hierarchical Self-Organizing Maps (GHSOM) often fail at continual learning because they tend to forget old information as they adapt to new data streams.
Methods
The authors propose two nested solutions:
- CAEA (CIM-based ART with Edge and Age): This is the base "flat" clustering algorithm. It uses the Correntropy-Induced Metric (CIM) instead of standard Euclidean distance, which makes it much more robust against outliers. Crucially, it uses a Kernel Density Estimator (KDE) to automatically calculate the similarity threshold based on the distribution of the first few data points, removing the need for manual "vigilance" tuning.
- HCAEA (Hierarchical CAEA): This is a divisive hierarchical structure. It starts with one CAEA model. Once that model is trained, the data points assigned to each node are passed down to "child" CAEA models in a new layer. This process repeats, building a tree-like structure that captures information at multiple levels of granularity.
The system handles continual learning by using ART’s logic: if a new data point doesn't fit any existing cluster (based on the automated threshold), a new node is created. Old nodes are preserved, preventing the "catastrophic forgetting" seen in other neural networks.
Datasets
The researchers evaluated their approach using a comprehensive mix of benchmarks:
- Synthetic Datasets: Aggregation, Compound, Hard Distribution, Jain, and Pathbased (used to test the ability to handle complex shapes).
- Real-World Datasets: 9 datasets from the UCI repository, ranging from low-dimensional (Iris, Wine) to high-dimensional (COIL20 with 1,024 attributes and TOX171 with 5,748 attributes).
Key Insights and Ideas
- Automation is Possible: By using KDE to estimate bandwidth ($\sigma$), the algorithm can "sense" the density of the data and set its own sensitivity. This makes the model much more "plug-and-play" across different domains.
- Divisive is Better for Continual Learning: Unlike agglomerative clustering (which builds from the bottom up and usually requires the whole dataset at once), this divisive (top-down) approach can adaptively generate new partitions as the data distribution shifts over time.
- Outlier Rejection: Using Correntropy (CIM) allows the model to ignore "noise" that would typically pull a standard K-means cluster center off-target.
Key Results
The authors compared HCAEA against state-of-the-art models like GH-EXIN and HFTCA.
- Performance: In both stationary and non-stationary (changing) environments, HCAEA consistently ranked among the top performers. Statistical analysis (Friedman and Nemenyi tests) showed that HCAEA and GH-EXIN were the superior models overall.
- Stability: In non-stationary tests (where classes are introduced one by one), HCAEA maintained high accuracy on early classes even after learning later ones, proving its capability for continual learning.
- Efficiency: While GH-EXIN often generated a massive number of nodes (thousands in some cases), HCAEA maintained a much more compact and interpretable structure while achieving similar or better accuracy.
Limitations and Future Work
The authors identify a few areas for improvement:
- Node Explosion: On specific datasets like "Iris" or "Pathbased," the algorithm occasionally generates more nodes than necessary. They suggest a future mechanism to prune or delete redundant nodes.
- Initial Sample Dependency: The automated threshold relies on the distribution of the first $\lambda/2$ data points. If the very first batch of data isn't representative of the whole, the initial threshold might be suboptimal.
- Concept Drift: While the model handles new classes well, the authors want to further explore how it handles "concept drift"—where the definition of an existing class changes over time.
Summary updated