Phenology Alignment Network: A Novel Framework for Cross-Regional Time Series Crop Classification

Ziqiao Wang, Hongyan Zhang, Wei He, Liangpei ZhangWorkshops @ CVPR 2021 · OpenReview

Abstract

Timely and accurate crop type classification is essential for agricultural applications. However, cross-regional classification faces significant challenges due to phenological discrepancies between training and test regions. We propose the Phenology Alignment Network (PAN), a framework based on deep recurrent networks and unsupervised domain adaptation (DA). PAN utilizes a Temporal Spatial Network (TSNet) backbone with stacked RNNs and self-attention to extract multi-level features. By aligning hierarchical deep features from source and target domains, PAN reduces regional discrepancies without requiring target labels. Experiments on a new Sentinel-2 dataset across three Chinese study sites show that PAN improves overall accuracy by 5.62% and macro average F1 score by 0.094 over non-adapted baselines.

Introduction: The Challenge of Cross-Regional Classification

Crop classification using satellite imagery is vital for food security and policy, but it suffers from a major generalization problem. A model trained to recognize wheat in one province often fails in another due to phenology discrepancy. The same crop grows at different rates and exhibits different spectral signatures depending on local climate, soil, and planting traditions. Additionally, varying cloud cover introduces noise. Traditionally, researchers must manually label new data for every region, a process that is expensive and labor-intensive.

Datasets and Study Areas

The researchers introduced a large-scale benchmark dataset based on Sentinel-2 satellite imagery, comprising 2.33 million labeled samples across three distinct agricultural plains in China: Jianghan (Site A), Chengdu (Site B), and Wuhu (Site C).

Map of the three study sites in China: Sichuan (Site B), Hubei (Site A), and Anhui (Site C) provinces.

These sites vary significantly in elevation, field size, and planting density. The dataset includes classes for winter wheat, winter rape, fallow, and "other."

Visual comparison of study sites and their corresponding temporal NDVI curves for wheat and rape, highlighting phenological discrepancies.

Methodology: Phenology Alignment Network (PAN)

The proposed solution consists of two primary components:

  1. TSNet (Temporal Spatial Network): This backbone classifier uses stacked Gated Recurrent Units (GRUs) to capture temporal dependencies of crop growth. A self-attention mechanism focuses on critical phenological stages (e.g., heading or ripening) while filtering out noise.
  2. PAN (Phenology Alignment Network): This adaptation framework employs a "Siamese" architecture with two identical TSNet branches—one for the labeled source region and one for the unlabeled target region.
  3. Feature Alignment: Maximum Mean Discrepancy (MMD) loss minimizes the statistical distance between features from both regions.
  4. Hierarchical Strategy: PAN aligns both "middle-level" features (from GRU layers) and "high-level" features (from the attention module), ensuring the model learns region-agnostic representations of crop development.

Key Insights and Results

The core idea is treating spectral time-series data as a "growth curve." While the timing of growth shifts between regions, the structural evolution remains similar enough for mathematical alignment. The self-attention module acts as a smart filter, identifying unique spectral signatures during specific growth phases.

Key results include:

  • Accuracy Boost: PAN improved overall accuracy (OA) by an average of 5.62% and the macro F1 score by 0.094 across six transfer scenarios.
  • Performance: PAN achieved an average OA of 85.70%, outperforming Random Forests (82.49%), TempCNN (80.89%), and standard Transformers (77.26%).
  • Visual Evidence: t-SNE visualizations confirmed that PAN successfully separated crop clusters that were previously overlapping before adaptation.

Conclusion and Future Work

PAN demonstrates that unsupervised domain adaptation can effectively bridge the gap in cross-regional crop classification. While aligning multiple layers adds computational overhead, the "middle-and-high" alignment strategy provides an optimal balance. Future work will focus on scaling this framework to national or global levels to create a universal crop mapping system.

Summary updated

Abstract

Timely and accurate crop type classification plays an essential role in the study of agricultural application. However, large area or cross-regional crop classification confronts huge challenges owing to dramatic phenology discrepancy among training and test regions. In this work, we propose a novel framework to address these challenges based on deep recurrent network and unsupervised domain adaptation (DA). Specifically, we firstly propose a Temporal Spatial Network (TSNet) for pixelwise crop classification, which contains stacked RNN and self-attention module to adaptively extract multi-level features from crop samples under various planting conditions. To deal with the cross-regional challenge, an unsupervised DA-based framework named Phenology Alignment Network (PAN) is proposed. PAN consists of two branches of two identical TSNet pre-trained on source domain; one branch takes source samples while the other takes target samples as input. Through aligning the hierarchical deep features extracted from two branches, the discrepancy between two regions is decreased and the pre-trained model is adapted to the target domain without using target label information. As another contribution, a time series dataset based on Sentinel-2 was annotated containing winter crop samples collected on three study sites of China. Cross-regional experiments demonstrate that TSNet shows comparable accuracy to state-of-the-art methods, and PAN further improves the overall accuracy by 5.62%, and macro average F1 score by 0.094 unsupervisedly.
BibTeX
@inproceedings{wang2021phenology,
  author = {Ziqiao Wang and Hongyan Zhang and Wei He and Liangpei Zhang},
  title = {{Phenology Alignment Network: A Novel Framework for Cross-Regional Time Series Crop Classification}},
  booktitle = {Workshops @ CVPR},
  year = {2021},
  url = {https://openreview.net/forum?id=AZVIbPa9lXH},
}
Figure