CL-CaGAN: Capsule Differential Adversarial Continual Learning for Cross-Domain Hyperspectral Anomaly Detection

Jianing Wang, Siying Guo, Zheng Hua, Runhu Huang, Jinyu Hu, Maoguo Gong

TL;DR

This paper introduces CL-CaGAN, a novel deep learning framework for hyperspectral anomaly detection (HAD) that addresses the critical challenge of catastrophic forgetting in cross-domain scenarios. By combining a capsule network with a generative adversarial network (GAN) and incorporating continual learning strategies like sample replay and self-distillation, CL-CaGAN effectively learns to detect anomalies across different datasets without losing previously acquired knowledge.

What Problem Does This Paper Address?

Hyperspectral anomaly detection (HAD) aims to identify rare pixels with distinct spectral signatures within hyperspectral images (HSIs). While deep learning (DL) methods have shown promise, they typically struggle in "open-world" scenarios where new datasets or domains are encountered. This is primarily due to two issues: 1) limited prior information about anomalies and backgrounds, and 2) the "catastrophic forgetting" problem, where a model trained on new data completely loses its ability to perform on previously learned tasks. This paper tackles these challenges by proposing a continual learning approach that allows HAD models to adapt to new domains without forgetting past knowledge, making them more practical for real-world applications.

Methods

The core of the proposed method is the CL-CaGAN (Capsule Differential Adversarial Continual Learning for Cross-Domain Hyperspectral Anomaly Detection) framework. Key components include:

  • Capsule Network (CapsNet) integrated into GAN: A modified capsule network is used within the GAN's generator and discriminator. CapsNets are adept at capturing spatial relationships and orientation information, which is crucial for robustly estimating background distributions and distinguishing anomalies.
  • Adversarial Learning (GAN): The GAN framework is employed to learn the background distribution. The generator attempts to reconstruct background samples, while the discriminator tries to distinguish between real and generated background data.
  • Continual Learning (CL) Strategies:
    • Clustering-based Sample Replay: To combat catastrophic forgetting, a replay buffer stores representative background samples from previous tasks. These samples are selected using K-means clustering to ensure they capture the diversity of past data.
  • Self-Distillation Regularization: An additional loss term encourages the current model's outputs to approximate those of the previous model, further reinforcing learned knowledge and preventing forgetting.
  • Differentiable Data Augmentation: This technique is applied to both real and generated data to stabilize the training process, improve convergence, and enhance the reconstruction of background samples.

Datasets

The paper evaluates the proposed CL-CaGAN on five real hyperspectral image (HSI) datasets:

  • Los Angeles-1
  • Los Angeles-2
  • Cat Island
  • San Diego
  • Bay Champagne

These datasets were captured by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor and represent different scenarios, allowing for the assessment of cross-domain performance.

Key Insights and Ideas

  • Capsules for better background modeling: Integrating CapsNets into the GAN framework significantly improves the ability to capture nuanced spectral-spatial characteristics of background, leading to more accurate anomaly detection.
  • Continual learning is essential for real-world HAD: Traditional DL models are ill-suited for evolving environments. CL strategies are vital for creating robust HAD systems that can adapt over time.
  • Balancing new knowledge and old knowledge: The proposed CL-CaGAN effectively balances learning new information from incoming tasks with retaining knowledge from previous tasks, a critical aspect of lifelong learning.
  • The importance of sample selection: The clustering-based replay strategy is crucial for selecting informative background samples, preventing the replay buffer from becoming too large or too biased.

Key Results

  • Superior Anomaly Detection Performance: CL-CaGAN consistently outperforms traditional and other DL-based HAD methods across various metrics, including AUC (Area Under the ROC Curve), ACC (Average Accuracy), and BWT (Backward Transfer).
  • Effective Mitigation of Catastrophic Forgetting: The proposed CL-CaGAN demonstrates significantly better BWT scores, indicating its ability to retain performance on previous tasks even after learning new ones.
  • Robustness in Cross-Domain Scenarios: Experiments show that CL-CaGAN achieves higher and more stable detection performance when tested on sequences of different datasets, highlighting its adaptability.
  • Visual Detection Results: Qualitative results (detection maps) show that CL-CaGAN can detect anomalies more accurately and with fewer false alarms compared to other methods, especially in challenging datasets like Cat Island and Bay Champagne.

Limitations and Future Work

  • Computational Complexity: While not explicitly detailed as a limitation, GANs and continual learning methods can be computationally intensive.
  • Hyperparameter Tuning: The performance of CL-CaGAN, particularly the threshold $\beta$ for CBM generation and the number of clustering groups $k$, requires careful tuning.
  • Future Work: The authors suggest that their work paves the way for practical DL applications in open-scenario cross-domain anomaly detection. Further research could explore more advanced CL strategies or investigate the application of CL-CaGAN to other remote sensing tasks.

Summary updated

Abstract

Anomaly detection (AD) has attracted remarkable attention in hyperspectral image (HSI) processing fields, and most existing deep learning (DL)-based algorithms indicate dramatic potential for detecting anomaly samples through specific training process under current scenario. However, the limited prior information and the catastrophic forgetting problem indicate crucial challenges for existing DL structure in open scenarios cross-domain detection. In order to improve the detection performance, a novel continual learning-based capsule differential generative adversarial network (CL-CaGAN) is proposed to elevate the cross-scenario learning performance for facilitating the real application of DL-based structure in hyperspectral AD (HAD) task. First, a modified capsule structure with adversarial learning network is constructed to estimate the background distribution for surmounting the deficiency of prior information. To mitigate the catastrophic forgetting phenomenon, clustering-based sample replay strategy and a designed extra self-distillation regularization are integrated for merging the history and future knowledge in continual AD task, while the discriminative learning ability from previous detection scenario to current scenario is retained by the elaborately designed structure with continual learning (CL) strategy. In addition, the differentiable enhancement is enforced to augment the generation performance of the training data. This further stabilizes the training process with better convergence and efficiently consolidates the reconstruction ability of background samples. To verify the effectiveness of our proposed CL-CaGAN, we conduct experiments on several real HSIs, and the results indicate that the proposed CL-CaGAN demonstrates higher detection performance and continuous learning capacity for mitigating the catastrophic forgetting under cross-domain scenarios.
BibTeX
@article{wang2024cl,
  author = {Jianing Wang and Siying Guo and Zheng Hua and Runhu Huang and Jinyu Hu and Maoguo Gong},
  title = {{CL-CaGAN: Capsule Differential Adversarial Continual Learning for Cross-Domain Hyperspectral Anomaly Detection}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2024},
  url = {https://openreview.net/forum?id=B9jXCmGpbH},
}
Figure