Transfer component analysis for domain adaptation in image classification

Giona Matasci, Michele Volpi, Devis Tuia, Mikhaïl KanevskiProceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE 2011 · DOI

Abstract

This contribution studies a feature extraction technique aiming at reducing differences between domains in image classification. The purpose is to find a common feature space between labeled samples issued from a source image and test samples belonging to a related target image. The presented approach, Transfer Component Analysis (TCA), finds a transformation matrix performing a joint mapping of the two domains by minimizing a probability distribution distance measure, the Maximum Mean Discrepancy (MMD) criterion. When predicting on a target image, such a projection allows to apply a supervised classifier trained exclusively on labeled source pixels mapped in this common latent subspace. Promising results are observed on an urban scene captured by a hyperspectral image, revealing improvements with respect to standard classification models and other feature extraction techniques.

Introduction

In the world of remote sensing and image classification, "ground truth" (manually labeled data) is the gold standard but is incredibly expensive and time-consuming to collect. Ideally, we would train a model on one image (the source domain) and use it to classify a new image (the target domain).

However, "dataset shift" makes this difficult. Even if two images contain the same materials (e.g., asphalt, grass, roofing), differences in sensor angles, solar elevation, shadows, or vegetation health mean the raw data values for those materials will look different to a computer. Standard classifiers fail because the statistical distribution of the source data doesn't match the target data. This paper explores how to bridge that gap using Domain Adaptation (DA) through Transfer Component Analysis.

Methodology: Transfer Component Analysis (TCA)

The authors propose using Transfer Component Analysis (TCA), a feature extraction technique specifically designed for domain adaptation. Unlike standard feature extraction that treats datasets in isolation, TCA performs a joint mapping of both domains into a common feature space.

Flowchart of the TCA-based domain adaptation process.

The process involves three key steps:

  1. Maximum Mean Discrepancy (MMD): To fix the shift, the distance between the means of the two distributions is calculated in a high-dimensional "kernel" space (RKHS).
  2. The TCA Algorithm: TCA seeks a transformation matrix ($W$) that minimizes the MMD between the source and target domains so they overlap in the new feature space, while simultaneously preserving the data's variance to keep classes discriminable.
  3. Classification: After projecting both datasets into this new "latent subspace," a standard Linear Discriminant Analysis (LDA) classifier is trained on the transformed source labels and tested on the transformed target data.

Experimental Setup

The study uses a well-known benchmark in the remote sensing community: the ROSIS-03 Hyperspectral Image of Pavia, Italy. The authors retained 102 spectral bands and focused on four classes: buildings, roads, shadows, and vegetation.

Spatial distribution of the Source A, Source B, and Target regions in the Pavia image.

To simulate real-world scenarios, the image was partitioned into three spatially disjoint sub-regions: "Source A" (moderate shift from target), "Source B" (large shift from target), and "Target." This allows for testing how well the model generalizes when training data comes from a different part of a city or a different flight line.

Key Insights and Results

  • Unlabeled Data is a Resource: TCA is "partially unsupervised." Even without labels for the target image, the shape of its unlabeled distribution provides vital information for finding a transformation that aligns it with the source.
  • Superior Accuracy: In the "large shift" scenario (Source B to Target), a model trained on raw source data failed significantly. TCA achieved notable accuracy levels, exceeding 90.50% Overall Accuracy (OA).
  • Closing the Gap: TCA consistently outperformed standard PCA and Kernel PCA. In the most challenging scenarios, TCA provided up to an 18% gain in Overall Accuracy over models that didn't account for the distribution shift, performing nearly as well as a model trained directly on target data.

Conclusion

TCA offers a robust framework for unsupervised domain adaptation in remote sensing. By creating a shared feature space that minimizes statistical divergence, it allows for the effective reuse of existing labeled datasets across different geographic regions or acquisition conditions, significantly reducing the need for expensive new ground truth collection.

Summary updated

Abstract

This contribution studies a feature extraction technique aiming at reducing differences between domains in image classification. The purpose is to find a common feature space between labeled samples issued from a source image and test samples belonging to a related target image. The presented approach, Transfer Component Analysis, finds a transformation matrix performing a joint mapping of the two domains by minimizing a probability distribution distance measure, the Maximum Mean Discrepancy criterion. When predicting on a target image, such a projection allows to apply a supervised classifier trained exclusively on labeled source pixels mapped in this common latent subspace. Promising results are observed on a urban scene captured by a hyperspectral image. The experiments reveal improvements with respect to a standard classification model built on the original source image and other feature extraction techniques.
BibTeX
@inproceedings{matasci2011transfer,
  author = {Giona Matasci and Michele Volpi and Devis Tuia and Mikhaïl Kanevski},
  title = {{Transfer component analysis for domain adaptation in image classification}},
  booktitle = {Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE},
  year = {2011},
  doi = {10.1117/12.898229},
}
Figure