A Hierarchical Multiclassifier System for Hyperspectral Data Analysis

Shailesh Kumar, Joydeep Ghosh, Melba M. CrawfordMultiple Classifier Systems 2000 · DOI

TL;DR

This paper introduces a novel hierarchical multiclassifier system that breaks down complex hyperspectral data classification into a series of simpler two-class problems. By automatically discovering a class hierarchy and learning specialized feature extractors at each step, the system achieves significantly higher classification accuracy and provides interpretable domain knowledge.

What Problem Does This Paper Address?

Classifying hyperspectral data, which contains information across hundreds of spectral bands, presents a significant challenge due to the "curse of dimensionality" – a high number of input features and a large number of potential classes. Traditional approaches often struggle with parameter estimation and computational complexity. This paper aims to simplify this problem by decomposing it into a hierarchy of manageable two-class problems, each with its own reduced feature space, and by learning a modular system that can automatically discover relationships between classes.

Methods

The core of the proposed system is a hierarchical multiclassifier architecture built as a binary tree. Each internal node in the tree represents a two-class problem, tasked with distinguishing between two "meta-classes" (groups of original classes). The key innovation lies in the coupled learning process at each node:

  • Partitioning: The algorithm automatically partitions the current set of classes into two disjoint meta-classes.
  • Feature Extraction: Simultaneously, it learns a linear feature extractor (specifically, a Fisher discriminant projection) that maximizes the discrimination between these two meta-classes in a reduced one-dimensional space.

This process is recursively applied until each leaf node represents a single original class. The learning framework is based on Generalized Associative Modular Learning Systems (GAMLS), which uses a deterministic annealing approach to gradually refine associations between samples and modules, leading to specialized classifiers. For classification, the posterior probabilities from each node are multiplied up the tree to obtain the final class posterior for a given input.

Datasets

The proposed method was evaluated on a hyperspectral dataset acquired by the NASA AVIRIS spectrometer over Kennedy Space Center in Florida. This dataset comprised 183 spectral bands (out of 224 available) and was used to classify 12 landcover types. For each class, approximately 350 examples were available, which were randomly split into 50% training and 50% test sets for 10 experiments.

Key Insights and Ideas

  • Decomposition is Key: Breaking down a complex C-class problem into C-1 sequential two-class problems is an effective strategy for managing high-dimensional data and large class sets.
  • Coupled Partitioning and Feature Extraction: Learning the optimal class partition and the best feature extractor simultaneously at each hierarchical level is crucial for maximizing discriminative power.
  • Automatic Domain Knowledge Discovery: The discovered class hierarchy aligns remarkably well with human expert opinions, suggesting that this modular learning approach can automatically uncover meaningful relationships and domain knowledge from data.
  • Dimensionality Reduction: The system achieves significant dimensionality reduction, projecting the 183-dimensional input space down to a single dimension at each internal node.

Key Results

  • Improved Classification Accuracy: The hierarchical multiclassifier achieved an overall classification accuracy of 97% on the test set, a significant improvement over:
  • A Bayesian pairwise classifier (93%).
  • A single MLP classifier (74.5%).
  • Other feature extraction methods like Principal Component Analysis and MNF transforms (at least 10% lower accuracy).
  • Significant Feature Reduction: The system reduced the 183-dimensional input space to a single dimension for classification at each internal node, leading to a substantial reduction in computational complexity and feature space size.
  • Interpretable Hierarchy: The automatically generated binary tree structure for the 12 landcover classes closely matched the grouping of classes by a human expert, validating the system's ability to learn domain-relevant structures.

Limitations and Future Work

  • Computational Cost: While the paper highlights improvements, the initial description of pairwise classifiers mentioned a potential O(C^2) complexity. The hierarchical approach mitigates this to O(C-1) classifiers, but the GAMLS framework's iterative learning process might still be computationally intensive.
  • Generalizability of GAMLS: The paper relies on the GAMLS framework for learning. While effective here, its broader applicability and scalability to even larger datasets or more complex class structures could be a point for future exploration.
  • Future Work: The authors suggest that the automatic decomposition into a hierarchy provides significant domain knowledge, implying that further analysis of these discovered relationships could be a fruitful area for future research. The paper also implicitly suggests that extending this approach to other complex classification problems beyond hyperspectral data could be beneficial.

Summary updated

BibTeX
@inproceedings{kumar2000hierarchical,
  author = {Shailesh Kumar and Joydeep Ghosh and Melba M. Crawford},
  title = {{A Hierarchical Multiclassifier System for Hyperspectral Data Analysis}},
  booktitle = {Multiple Classifier Systems},
  year = {2000},
  doi = {10.1007/3-540-45014-9\_26},
}
Figure