Clustering Approach to Solve Hierarchical Classification Problem Complexity

Aomar Osmani, Massinissa Hamidi, Pegah AlizadehAAAI 2022 · DOI

In Human Activity Recognition (HAR), distinguishing between similar activities like "walking" and "running" is a persistent challenge. These activities often exist on a spectrum, blurred by sensor noise and individual variability. While hierarchical classification—grouping similar activities before fine-tuning—is effective, it typically relies on human experts to manually design the category tree. However, for even a small number of classes, the number of possible tree structures is immense, making manual optimization nearly impossible.

A recent AAAI-22 paper, "Clustering Approach to Solve Hierarchical Classification Problem Complexity," introduces an automated framework that allows data to define its own hierarchy. This approach not only improves classification accuracy but also drastically reduces the volume of labeled data required for training.

Methodology: Automated Hierarchy Discovery

The core of the proposed framework is a recursive clustering process that identifies the most separable groups of concepts without human intervention.

Overview of the clustering-based framework for hierarchy discovery.

The process relies on two key metrics to evaluate and select the best clustering results:

  • Cohesion ($\phi_C$): Measures the frequency with which two classes are grouped together across multiple clustering runs, indicating a shared inductive bias.
  • Dispersion ($\psi_C$): Quantifies how much a single class is spread across different clusters; lower dispersion suggests a more distinct and stable category.

By recursively applying these metrics, the system builds a hierarchy where top-level nodes handle broad distinctions (e.g., "motorized" vs. "body-powered") and leaf nodes focus on fine-grained classification.

Training Specialized Learners

Once the hierarchy is established, the authors assign specialized classifiers to each node. These learners are trained using strategies inspired by meta-learning:

  • Boosting: Lower-level nodes focus on examples that were difficult for their parent nodes to classify.
  • Student-Teacher Learning: Knowledge is transferred down the tree, allowing descendant nodes to benefit from the broader context captured by their ancestors.

This hierarchical structure allows different parts of the model to inherit specific inductive biases relevant to their level of granularity. For instance, the features needed to distinguish "moving" from "still" differ significantly from those required to separate "bus" from "train."

Experimental Results and Data Efficiency

The researchers evaluated their approach on three major HAR benchmarks: SHL (Sussex-Huawei Locomotion), USC-HAD, and HTC-TMD.

Performance gains and data efficiency comparison.

The results demonstrate significant improvements over standard "flat" classification models:

  • Accuracy: On the SHL dataset, the proposed method achieved 73.7% accuracy, outperforming state-of-the-art models like DeepConvLSTM (65.3%) and DeepSense (66.5%).
  • Data Efficiency: The hierarchical model proved exceptionally robust in low-data scenarios. By leveraging the combined data of all descendants at higher nodes, the system achieved comparable or superior results while using significantly fewer training instances—up to $2 \times 10^{-3}$ fewer examples in certain settings.
  • Sensitivity Analysis: Using fANOVA, the authors identified "Stride" (convolutional step size) as the most critical hyperparameter, as it dictates the temporal resolution necessary for the model to distinguish between activities.

Conclusion

By automating the discovery of classification hierarchies through unsupervised clustering, this research bypasses the limitations of manual tree design. The resulting models are more accurate, more data-efficient, and better aligned with the underlying structure of sensor data. Future work aims to make the hierarchy discovery end-to-end trainable, potentially allowing the structure to evolve dynamically alongside the classifiers.

Summary updated

BibTeX
@inproceedings{osmani2022clustering,
  author = {Aomar Osmani and Massinissa Hamidi and Pegah Alizadeh},
  title = {{Clustering Approach to Solve Hierarchical Classification Problem Complexity}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i7.20760},
}
Figure