Abstract
This work develops a novel end-to-end deep unsupervised learning method based on convolutional neural network (CNN) with pseudo-classes for remote sensing scene representation. First, we introduce center points as the centers of the pseudo classes and the training samples can be allocated with pseudo labels based on the center points. Therefore, the CNN model, which is used to extract features from the scenes, can be trained supervised with the pseudo labels. Moreover, a pseudo-center loss is developed to decrease the variance between the samples and the corresponding pseudo center point. The pseudo-center loss is important since it can update both the center points with the training samples and the CNN model with the center points in the training process simultaneously. Finally, joint learning of the pseudo-center loss and the pseudo softmax loss which is formulated with the samples and the pseudo labels is developed for unsupervised remote sensing scene representation to obtain discriminative representations from the scenes. Experiments are conducted over two commonly used remote sensing scene datasets to validate the effectiveness of the proposed method and the experimental results show the superiority of the proposed method when compared with other state-of-the-art methods.
Introduction
High-resolution remote sensing imagery from satellites and aerial platforms is abundant, yet labeling these images for supervised learning is labor-intensive and requires significant domain expertise. Unsupervised learning offers a solution by extracting features without manual labels. However, remote sensing scenes present unique challenges: high intra-class variance (similar scenes looking different) and low inter-class variance (different scenes looking similar).
Existing unsupervised methods often rely on static, hand-crafted features or pre-defined pseudo-labels that do not evolve during training. This work proposes an end-to-end approach where the feature extractor and category definitions improve each other iteratively in a virtuous cycle.
Methodology
The proposed framework utilizes a Convolutional Neural Network (CNN) to learn discriminative representations through a joint learning process. The core innovation is the dynamic construction of pseudo-classes using center points in the feature space.
Pseudo-Class Construction and Labeling
The model maintains a set of "center points" representing pseudo-classes. For each training batch, images are assigned pseudo-labels based on their proximity to these centers. This process allows the model to create its own supervision signal dynamically.
Joint Loss Function
The network is optimized using a combination of two losses:
- Pseudo-Softmax Loss ($L_s$): This treats the generated pseudo-labels as ground truth to train the CNN to discriminate between clusters.
- Pseudo-Center Loss ($L_c$): This minimizes the distance between a sample's features and its assigned center point. Crucially, this loss updates both the CNN parameters and the positions of the center points themselves via backpropagation.
This joint optimization allows the "moving target" of cluster centers to refine alongside the feature extractor, preventing the model from stagnating on poor initial clusters and ensuring the features remain discriminative.
Experimental Evaluation
The method was validated on two widely recognized benchmarks: the UC Merced Land Use Dataset (21 classes) and the Brazilian Coffee Scene Dataset (2 classes).
Key Results
The proposed method demonstrated significant improvements over both traditional hand-crafted methods and other deep unsupervised approaches:
- UC Merced: Achieved 94.33% accuracy, outperforming Dense SIFT (81.67%), MARTA GANs (85.37%), and UCFFN (88.57%).
- Brazilian Coffee: Achieved 87.74% accuracy, surpassing standard CNN-1 models (86.00%) and SIFT (82.83%).
Analysis revealed that the tradeoff parameter $\lambda$ (balancing the two losses) is critical; the model performs best when there is a balance between collapsing samples into centers and maintaining discriminative power. Furthermore, the model often performed best when the number of pseudo-classes was smaller than the actual number of ground-truth classes, suggesting it effectively learns broad semantic groupings first.
Conclusion
This work demonstrates a powerful end-to-end unsupervised learning framework for remote sensing scene representation. By jointly optimizing feature extraction and cluster center refinement, the model achieves state-of-the-art accuracy without human labels. Future research will explore more advanced architectures like ResNet and the application of this joint learning framework to other complex tasks such as object detection.
Summary updated