Abstract
Illegal, Unreported, and Unregulated (IUU) fishing poses a significant threat to global marine ecosystems and economic stability. Traditional monitoring systems, such as the Automatic Identification System (AIS), are easily circumvented by "dark vessels" that disable their transponders. This paper presents xView3-SAR, the largest publicly available labeled dataset for maritime object detection using Synthetic Aperture Radar (SAR). By providing nearly 1,000 analysis-ready Sentinel-1 images and over 240,000 annotations, this work enables the development of machine learning models capable of global, all-weather maritime surveillance. We detail the dataset creation, the multitask challenge formulation, and the results of the xView3 Computer Vision Challenge, highlighting both the progress in vessel detection and the persistent difficulties of near-shore monitoring.
Introduction
Illegal, Unreported, and Unregulated (IUU) fishing accounts for more than 20% of the global catch, resulting in billions of dollars in economic losses and devastating impacts on ocean biodiversity. Effective enforcement is currently hampered by the limitations of the Automatic Identification System (AIS). While AIS is mandatory for large vessels, illicit actors frequently disable their broadcasts to become "dark vessels," effectively vanishing from standard tracking systems.
Satellite imagery offers a potential solution for monitoring these elusive actors. However, standard optical sensors are limited by cloud cover and the absence of daylight. Synthetic Aperture Radar (SAR) overcomes these limitations by providing high-resolution imagery regardless of weather or lighting conditions. Despite its potential, SAR has historically been difficult for the machine learning community to utilize due to the complexity of the data and a lack of large-scale, high-quality labeled datasets. The xView3-SAR dataset was designed to bridge this gap.
The Challenges of SAR Maritime Surveillance
Detecting vessels in SAR imagery involves overcoming two primary hurdles:
- Domain Complexity: SAR data is not a standard optical image; it is a record of radar backscatter. Transforming raw SAR data into "analysis-ready" formats requires specialized, computationally intensive preprocessing.
- Data Scarcity and Sparsity: Maritime objects are extremely sparse relative to the ocean surface. In the xView3-SAR dataset, vessels account for a mere 0.005% of total pixels, making detection a classic "needle in a haystack" problem.
The xView3-SAR Dataset
The primary contribution of this work is the xView3-SAR dataset, which represents a massive leap in scale over previous benchmarks. While the previous largest dataset (LS-SSDD-v1.0) contained 12 gigapixels of data, xView3-SAR provides 1,400 gigapixels.
Data Acquisition and Processing
The authors processed raw Sentinel-1 SAR data into 10-meter resolution GeoTIFFs. Each scene includes both VH (cross-polarization) and VV (co-polarization) bands. The VH band is particularly effective for distinguishing ships from sea clutter, while the VV band provides better texture for ocean surface characterization. To provide environmental context, each SAR image is paired with ancillary data, including bathymetry (ocean depth) and wind state rasters.
Hybrid Labeling Strategy
To ensure high-confidence labels for both cooperative and "dark" vessels, a hybrid approach was employed:
- AIS Matching: Known vessels were labeled using a state-of-the-art AIS-to-SAR matching algorithm.
- Expert Annotation: Professional maritime analysts manually identified and verified objects lacking an AIS signature, ensuring the dataset captures the very vessels that attempt to evade detection.
Multitask Learning Formulation
Rather than treating maritime surveillance as a simple object detection problem, the xView3-SAR challenge frames it as a multitask learning problem. Models are evaluated on their ability to perform:
- Detection: Locating maritime objects.
- Classification: Distinguishing between vessels and fixed infrastructure (e.g., oil rigs).
- Characterization: Identifying fishing vs. non-fishing vessels.
- Estimation: Regressing the physical length of the detected vessel.
Results and Insights
The xView3 Computer Vision Challenge demonstrated a significant performance leap over baseline methods. The top-performing model, "BloodAxe," achieved an aggregate multitask score of 0.6177, compared to the Faster-RCNN reference baseline of 0.1904.
Key Findings
- Classification Success: Top models achieved F1-scores exceeding 0.93 for distinguishing vessels from infrastructure, suggesting that machine learning can effectively automate the sorting of maritime objects.
- The Shoreline Struggle: A significant finding was the difficulty of detection near coastlines. Rough terrain and coastal rocks create radar artifacts that mimic the appearance of ships. No current model has fully resolved this "near-shore" false positive problem.
- Inference Efficiency: Due to the massive size of SAR scenes (averaging 29,400 x 24,400 pixels), the challenge imposed a 15-minute inference limit. This requirement favored efficient, single-stage architectures like CircleNet over more complex, multi-stage detectors.
Limitations and Future Work
While xView3-SAR is a landmark dataset, it has limitations. The geographic distribution is currently focused on European and West African waters, and very small, non-metallic boats remain difficult to detect as they often blend into radar noise (speckle).
Future research should focus on improving shoreline delineation and exploring multi-temporal data—analyzing the same geographic area over time—to better distinguish small vessels from transient radar noise. By providing this dataset to the public, the authors have established a foundation for a new era of automated, global maritime transparency.
Summary updated