Robust Semi-Supervised Classification using GANs with Self-Organizing Maps

Ronald Fick, Paul D. Gader, Alina ZarearXiv 2021 · arXiv

Abstract

This paper presents a novel architecture combining Generative Adversarial Networks (GANs) with Self-Organizing Maps (SOMs) to address the "Discriminating Outliers from Inliers while maintaining Classification accuracy" (DOIC) problem. In semi-supervised learning (SSL), unlabeled data often contains "outliers" not belonging to any target class. By integrating SOM-generated "competency" features into a dual-path GAN discriminator, the proposed model significantly enhances outlier detection and classification performance, particularly on complex hyperspectral datasets.

Introduction

Semi-supervised learning (SSL) leverages small labeled datasets alongside large unlabeled pools. Generative Adversarial Networks (GANs) are frequently used for this, where a discriminator classifies data into $K$ classes or identifies it as "fake." However, standard SS-GANs assume all unlabeled data belongs to the $K$ inlier classes. In real-world applications like remote sensing, unlabeled data is often "dirty," containing outliers. Forcing these into inlier categories reduces model reliability—a challenge the authors term the DOIC problem.

Methodology

The proposed architecture integrates a Self-Organizing Map (SOM) layer into the GAN framework to provide a low-dimensional representation of the input space.

  1. SOM Feature Layer: Data is processed by a SOM, and distances between input samples and every node are calculated.
  2. Custom Distance Metrics: For hyperspectral data, the authors use a combination of Mahalanobis distance and Spectral Angle to ensure robustness against illumination changes.
  3. Membership Mapping: These distances are converted into "membership values" via a sigmoid function. Inliers map to high values for specific nodes, while outliers result in low membership across the entire map.
  4. Dual-Path Discriminator: The discriminator features two paths: one for raw spectral features (classification) and one for SOM membership features (outlier detection). The outputs are concatenated for a joint decision.
SOM Weights showing learned spectral signatures across the grid.

Datasets

The researchers evaluated their approach on two challenging hyperspectral datasets:

  • MUUFL Gulfport: Features vegetation and man-made materials; the goal was to classify trees/grass while treating roads and buildings as outliers.
  • Santa Barbara AVIRIS: A complex dataset with 30 classes (mostly tree species), providing a rigorous test for both classification and outlier detection.

Key Insights and Ideas

The SOM acts as a "competency-aware" layer, determining if the model recognizes the data type before attempting classification. This synergy between SSL and SOMs allows the generator to learn the data distribution more effectively, which in turn helps the discriminator refine its understanding of what constitutes an "inlier."

SOM activation heatmaps for different tree species, demonstrating how classes are localized on the map.

Key Results

The inclusion of SOM features led to a massive leap in performance:

  • Outlier Detection: On the MUUFL dataset, ROC AUC increased from ~0.5-0.6 (spectral only) to ~0.99 with SOM features, representing near-perfect discrimination.
  • Classification Accuracy: The SS-GAN + SOM approach achieved 10-12% higher accuracy than purely supervised methods on the Santa Barbara dataset.
  • Visual Robustness: Qualitative results showed that while standard GANs often mislabeled non-target materials as inliers, the SOM-augmented models correctly identified them as outliers.

Conclusion

While highly effective for hyperspectral data, future work should explore the SOM-GAN architecture's performance on RGB imagery and address the computational overhead of the dual-path discriminator. End-to-end training of the SOM and GAN remains a promising research direction.

Summary updated

Abstract

Generative adversarial networks (GANs) have shown tremendous promise in learning to generate data and effective at aiding semi-supervised classification. However, to this point, semi-supervised GAN methods make the assumption that the unlabeled data set contains only samples of the joint distribution of the classes of interest, referred to as inliers. Consequently, when presented with a sample from other distributions, referred to as outliers, GANs perform poorly at determining that it is not qualified to make a decision on the sample. The problem of discriminating outliers from inliers while maintaining classification accuracy is referred to here as the DOIC problem. In this work, we describe an architecture that combines self-organizing maps (SOMs) with SS-GANS with the goal of mitigating the DOIC problem and experimental results indicating that the architecture achieves the goal. Multiple experiments were conducted on hyperspectral image data sets. The SS-GANS performed slightly better than supervised GANS on classification problems with and without the SOM. Incorporating the SOMs into the SS-GANs and the supervised GANS led to substantially mitigation of the DOIC problem when compared to SS-GANS and GANs without the SOMs. Furthermore, the SS-GANS performed much better than GANS on the DOIC problem, even without the SOMs.
BibTeX
@misc{fick2021robust,
  author = {Ronald Fick and Paul D. Gader and Alina Zare},
  title = {{Robust Semi-Supervised Classification using GANs with Self-Organizing Maps}},
  year = {2021},
  eprint = {2110.10286},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.10286},
}
Figure