Abstract
Annotating satellite imagery is a resource-intensive task requiring significant domain expertise. While self-supervised learning (SSL) has mitigated some labeling burdens, current approaches often overlook the fundamental importance of material and texture. We introduce MATTER (MATerial and TExture Representation Learning), a framework that explicitly targets these low-level features. By utilizing multi-temporal imagery of stable regions, MATTER learns invariant surface representations. Our results demonstrate state-of-the-art performance across multiple benchmarks, including a 76% faster convergence in building segmentation tasks.
Introduction
The challenge of remote sensing lies in the complexity of the data. Distinguishing between a concrete roof and a paved road, or identifying subtle land-use changes, requires more than just high-level semantic understanding; it requires an appreciation of the physical "fabric" of the Earth. Traditional SSL methods in this domain typically leverage geographic location or temporal cycles, which may ignore the micro-structures defining surface materials. MATTER addresses this by introducing an inductive bias focused on texture and material properties, facilitating more efficient downstream learning for tasks like change detection and urban mapping.
Methodology: The MATTER Framework
The MATTER architecture is designed to prioritize low-level surface properties through two primary components:
- Texture Refinement Network (TeRN): Deep neural networks often discard fine-grained details in favor of global context. TeRN counters this by employing a deterministic, pixel-adaptive convolution layer. This layer amplifies micro-structures based on local pixel similarity, ensuring that the model retains critical texture information early in the encoding process.
- Surface Residual Encoder: Drawing inspiration from classical "texton" methods, this module represents image patches as residuals relative to learned cluster centers. Instead of a monolithic feature vector, patches are characterized by their deviation from prototypical materials (e.g., grass, soil, or concrete).
The training strategy employs a contrastive learning objective (Noise Contrastive Estimation). By comparing spatially aligned images of the same unchanged location captured at different times, the model learns to produce consistent material representations despite variations in lighting, shadows, or sensor angles.
Experimental Setup
To train MATTER, a new pre-training dataset was curated consisting of 14,857 Sentinel-2 tiles from 27 globally distributed regions. These areas were specifically chosen for their lack of human development to ensure temporal stability. The model's performance was evaluated on three major benchmarks:
- OSCD (Onera Satellite Change Detection): Testing semantic change detection.
- BigEarthNet: A large-scale multi-label land cover classification dataset.
- SpaceNet: Focused on building footprint segmentation.
Results and Discussion
MATTER demonstrates significant improvements in both efficiency and accuracy. In building footprint segmentation (SpaceNet), the model reached convergence in just 24 epochs—a 76% reduction compared to the 100 epochs required by competing SSL methods.
On the OSCD change detection task, MATTER achieved a 24.22% improvement in F-1 score in unsupervised settings and a 6.33% improvement when fine-tuned, outperforming standard ImageNet-initialized models. These results suggest that material invariance is a powerful foundation for remote sensing. A key insight from our ablation studies is the "Receptive Field Paradox": while many vision tasks benefit from larger context, material recognition thrives on smaller receptive fields that prevent the model from ignoring local micro-structures.
Conclusion
MATTER proves that focusing on the "fabric" of the Earth—its materials and textures—provides a superior inductive bias for remote sensing tasks. By mastering these low-level properties, models can more easily transition to high-level semantic tasks. Future work will explore the application of this framework to non-optical data, such as Synthetic Aperture Radar (SAR), and investigate automated methods for curating stable pre-training datasets at a global scale.
Summary updated