Abstract
Remote sensing research often lacks high-resolution, labeled multispectral data suitable for training modern deep learning models. Furthermore, common evaluation practices frequently suffer from spatial correlation between training and testing sets. This paper presents a new benchmark dataset from Hamlin Beach State Park with a 4.7 cm ground-sample distance (GSD) across six spectral bands. By establishing fixed geographic splits for training, validation, and testing, we provide a rigorous framework for evaluating semantic segmentation models in remote sensing.
Introduction
While RGB-based computer vision has advanced rapidly due to large-scale datasets, the remote sensing community faces significant hurdles. Existing multispectral (MSI) and hyperspectral (HSI) datasets often feature low spatial resolution (1–20 m GSD), making small object identification impossible. Moreover, the practice of randomly sampling pixels from a single image for training and testing leads to "spatial contamination," where high correlation between neighboring pixels artificially inflates performance metrics.
Data Collection and Methods
Data was acquired at Hamlin Beach State Park using a DJI-S1000 octocopter. The platform was equipped with a Tetracam Micro-MCA6 sensor, capturing six spectral bands in the visible and near-infrared (VNIR) range (490–900 nm).
To establish performance baselines, several classification strategies were evaluated:
- Spectral-Only Classifiers: k-Nearest Neighbor (kNN), Support Vector Machines (SVM), and Multi-Layer Perceptrons (MLP).
- Spatial-Spectral Classifiers: Multiscale Independent Component Analysis (MICA) and Stacked Convolutional Autoencoders (SCAE).
- Unsupervised Feature Learning: Given the limited labeled data, unsupervised techniques were used to learn low-level filters (e.g., edge detectors) prior to classification.
The Hamlin Beach State Park Dataset
The resulting dataset offers a 4.7 cm GSD, providing unprecedented detail compared to satellite imagery. It includes 18 unbalanced classes, ranging from large-scale features like Lake Ontario to small targets like picnic tables and buoys. Crucially, the dataset is partitioned into three geographically distinct folds to prevent data leakage.
Results and Discussion
Experimental results highlighted several key findings:
- Generalization over Memorization: The geographic splits proved challenging, revealing that models often fail to generalize when spatial correlation is removed.
- The Value of NIR: Including near-infrared bands significantly improved segmentation accuracy, particularly for vegetation-heavy scenes.
- Model Performance: The unsupervised MICA approach achieved the highest Overall Accuracy (74.9%) and Mean-Class Accuracy (36.2%). Interestingly, MICA outperformed the deeper SCAE model, suggesting that for high-resolution MSI data with limited samples, robust low-level features over larger receptive fields may be more effective than complex deep hierarchies.
- Target Detection: The dataset also serves as a benchmark for finding specific targets (black and white wooden panels), where Spectral Matched Filter (SMF) and Adaptive Cosine Estimator (ACE) achieved AUC scores above 0.93.
Conclusion
The Hamlin Beach dataset provides a much-needed high-resolution benchmark for the remote sensing community. By enforcing geographic separation, it sets a higher standard for model evaluation. Future work will focus on developing end-to-end Deep Convolutional Neural Network (DCNN) architectures tailored for multispectral data and expanding the dataset to include more diverse ecological environments.
Summary updated