CDG: Conditional Domain Generalization for Hyperspectral Imagery Classification with Convergence and Constrained-risk Theories

Zhe Gao, Bin Pan, Zhenwei ShiarXiv 2024 · OpenReview

TL;DR

This paper introduces C³DG, a novel method for hyperspectral image classification that tackles the "hyperspectral-monospectra" problem (different materials having similar spectral signatures) by focusing solely on spectral data and employing a Conditional Revising Inference Block (CRIB). The method is theoretically grounded with convergence and risk bound theorems, demonstrating improved generalization across different domains.

What Problem Does This Paper Address?

Hyperspectral Imagery (HSI) classification is powerful due to the rich spectral information it captures, allowing for precise material identification. However, a significant hurdle is the "hyperspectral-monospectra" phenomenon, where distinct classes exhibit very similar spectral signatures. While combining spatial and spectral features has been a common approach, it can lead to inflated accuracy by inadvertently including test data in training patches. Existing domain generalization (DG) methods, though promising, often struggle to differentiate between similar spectra from varying domains and typically lack theoretical underpinnings. This paper aims to address these limitations by proposing a spectral-only DG method that can produce distinct outputs for similar inputs from different domains, backed by theoretical guarantees.

Methods

The core innovation is the Conditional Revising Inference Block (CRIB). CRIB acts as a domain feature extractor that can be integrated with any HSI classification backbone. It utilizes a shared encoder and multiple domain-specific decoders to effectively learn from the conditional distribution of the input data. This allows for a "decoupling" that aligns with how HSI data is generated.

The method is inspired by "test-time training" approaches, where models adapt at inference time. CRIB processes input spectra to extract domain variables and mixed variables. A pseudo-label classifier is used to guide the process, activating specific decoder branches based on rough class predictions. The outputs from these branches are then combined with the original input features to generate revised features for the backbone classifier.

Crucially, the authors provide theoretical justifications for their approach. They present:

  • Optimization Convergence Theorem: This theorem ensures that the gradients of the different loss terms within the model are not contradictory, guaranteeing that the model can converge during training.
  • Risk Upper Bound Theorem: This theoretical analysis establishes a concrete bound on the classification error, linking test-time training concepts with recent work on sharpness-aware minimization.

Datasets

The proposed C³DG method was evaluated on three benchmark HSI datasets:

  • Cross-Scene Houston: This dataset consists of two scenes from Houston (2013 and 2018) with varying numbers of spectral bands.
  • Cross-Scene Pavia: This dataset includes scenes from Pavia University and Pavia Center, captured by the ROSIS sensor.
  • Hyrank Datasets: This dataset comprises imagery from Dioni and Loukia regions, captured by the Hyperion sensor.

The paper also provides detailed information on the sample sizes for each class within these datasets.

Key Insights and Ideas

  • Spectral-Only DG is Viable: The paper demonstrates that focusing solely on spectral information can be sufficient to overcome the hyperspectral-monospectra problem, challenging the common reliance on spatial features.
  • CRIB for Adaptive Feature Extraction: The CRIB architecture is a clever way to leverage domain-specific information without drastically increasing model complexity, by sharing a common encoder and using class-specific decoders.
  • Theoretical Guarantees Matter: Providing convergence and risk bound theorems adds significant credibility and understanding to the proposed method, moving beyond empirical results.
  • Test-Time Adaptation for DG: The inspiration from test-time training highlights a powerful paradigm for adapting models to unseen domains at inference time.

Key Results

Experimental results on the three benchmark datasets consistently show that C³DG outperforms existing methods, including ERM (Empirical Risk Minimization) and various single-source and multi-source domain generalization approaches. For instance, on the Cross-Scene Houston dataset, C³DG achieved an Overall Accuracy (OA) of 68.23%, surpassing other methods. Similar superior performance was observed on the Pavia and Hyrank datasets, with C³DG often achieving the best OA, AA, and Kappa Coefficient scores. The paper also includes ablation studies that validate the effectiveness of each component of C³DG, particularly the CRIB and its conditional distribution learning capabilities.

Limitations and Future Work

The paper acknowledges that while C³DG shows superior performance, it is primarily focused on spectral information. Future work could explore integrating spatial information in a more principled way that complements the spectral-only approach. Additionally, the theoretical analysis, while robust, could be extended to cover more complex DG scenarios or different network architectures. The authors also suggest that their proposed conditional domain generalization strategy can be applied to a wider range of HSI classification methods, indicating potential for broader impact.

Summary updated

Abstract

Hyperspectral imagery (HSI) classification may suffer the challenge of hyperspectral-monospectra, where different classes present similar spectra. Joint spatial-spectral feature extraction is a popular solution for the problem, but this strategy tends to inflate accuracy since test pixels may exist in training patches. Domain generalization methods show promising potential, but they still fail to distinguish similar spectra across varying domains, in addition, the theoretical support is usually ignored. In this paper, we only rely on spectral information to solve the hyperspectral-monospectra problem, and propose a Convergence and Error-Constrained Conditional Domain Generalization method for Hyperspectral Imagery Classification (C$^3$DG). The major contributions of this paper include two aspects: the Conditional Revising Inference Block (CRIB), and the corresponding theories for model convergence and generalization errors. CRIB is the kernel structure of the proposed method, which employs a shared encoder and multi-branch decoders to fully leverage the conditional distribution during training, achieving a decoupling that aligns with the generation mechanisms of HSI. Moreover, to ensure model convergence and maintain controllable error, we propose the optimization convergence theorem and risk upper bound theorem. In the optimization convergence theorem, we ensure the model convergence by demonstrating that the gradients of the loss terms are not contradictory. In the risk upper bound theorem, our theoretical analysis explores the relationship between test-time training and recent related work to establish a concrete bound for error. Experimental results on three benchmark datasets indicate the superiority of C$^3$DG.
BibTeX
@misc{gao2024cdg,
  author = {Zhe Gao and Bin Pan and Zhenwei Shi},
  title = {{CDG: Conditional Domain Generalization for Hyperspectral Imagery Classification with Convergence and Constrained-risk Theories}},
  year = {2024},
  eprint = {2407.04100},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2407.04100},
}
Figure