Efficient Naval Surveillance: Addressing Label Noise with Rockafellian Risk Minimization for Water Security

Gabriel Custódio Rangel, Vander Alves, Igor Pınheıro de Araújo Costa, Miguel Ângelo Lellis Moreira, Arthur Pinheiro de Araújo Costa, Marcos dos Santos, Eric EckstrandWater 2025 · DOI

TL;DR

This paper introduces Rockafellian Risk Minimization (RRM), a novel machine learning approach designed to improve the accuracy of naval surveillance systems, particularly in the presence of noisy or corrupted image labels. By intelligently adjusting data point probabilities, RRM effectively mitigates the negative impact of label noise, leading to more robust and reliable ship detection models for maritime security.

What Problem Does This Paper Address?

Maritime environments are critical for water resource management and are unfortunately prone to illicit activities like illegal fishing, trafficking, and piracy. Effective surveillance of these vast areas relies heavily on computer vision and machine learning models for tasks like ship detection. However, real-world datasets used to train these models are often plagued by "label noise"—incorrectly assigned labels due to random errors or deliberate attacks. This noise significantly degrades the performance of traditional machine learning models, reducing their accuracy and reliability. Existing methods often require extensive manual data cleaning, which is time-consuming and impractical for large-scale maritime operations. This paper addresses the challenge of building robust ship detection models that can perform accurately even when trained on noisy datasets, without the need for meticulous pre-cleaning.

Methods

The core of this research is the Rockafellian Risk Minimization (RRM) framework, which is applied to train neural networks. RRM builds upon the concept of Rockafellian relaxation, a technique that allows for flexibility in decision-making when dealing with label noise. Unlike standard Empirical Risk Minimization (ERM), which aims to minimize the average loss across all training data, RRM introduces auxiliary variables (u-vectors) that dynamically adjust the probability assigned to each data point.

The RRM algorithm is implemented using an Alternating Direction Heuristic (ADH-LP). This iterative process involves two main steps:

  1. w-optimization: Standard neural network training (using SGD or Adam optimizers) to adjust network weights based on current data probabilities.
  2. u-optimization: A linear programming step that adjusts the u-vectors. This step aims to identify and effectively "exclude" potentially mislabeled data points from the training process by assigning them very low probabilities.

This two-step process allows the model to learn from the majority of correctly labeled data while down-weighting or ignoring corrupted samples, thereby improving overall model robustness and accuracy.

Datasets

The study utilizes two publicly available datasets for evaluating their proposed method:

  • MASATI Dataset: A collection of maritime scenes captured via optical aerial imagery, featuring a variety of weather and lighting conditions. For this study, a subset was created for binary classification (ship vs. sea), containing 2049 images.
  • AIRBUS Ship Detection Dataset: A large-scale dataset of satellite imagery with ship annotations, originally released for a Kaggle competition. The study used a balanced subset of 10,428 images, divided equally into "ship" and "no ship" categories.

Both datasets were preprocessed by resizing images to a common input dimension (128x128x3) to ensure consistency for the CNN architecture. Label noise was artificially introduced by randomly swapping training example labels at various percentages (10% to 60%).

Key Insights and Ideas

  • Label Noise is a Major Hurdle: The paper strongly emphasizes that label noise is a pervasive issue in real-world datasets, significantly impacting the performance of even sophisticated deep learning models.
  • RRM Offers a Robust Alternative to ERM: RRM demonstrates a clear advantage over traditional ERM, especially in scenarios with moderate to high levels of label corruption. It achieves comparable or superior accuracy without requiring extensive manual data cleaning.
  • The Power of Probability Adjustment: The core innovation lies in RRM's ability to dynamically adjust the influence of individual data points. By down-weighting potentially mislabeled samples, the model can learn more effectively from the reliable data.
  • Penalty Parameter (θ) is Crucial: The effectiveness of RRM is sensitive to the choice of the penalty parameter (θ). Finding the right balance is key to effectively excluding noisy data without discarding too much valuable information.
  • SGD Optimizer Shows Strong Resilience: While both Adam and SGD optimizers were tested, the SGD optimizer, when paired with RRM and the ADH-LP architecture, consistently showed remarkable resilience to increasing data corruption levels.

Key Results

The study evaluated RRM against ERM on both the MASATI and AIRBUS datasets under varying levels of label contamination.

  • MASATI Dataset (with Adam optimizer): RRM consistently matched or outperformed ERM, especially with optimal θ values between 0.15 and 0.25. At 40% corruption, RRM achieved accuracy comparable to ERM's performance at 0% corruption.
  • MASATI Dataset (with SGD optimizer): RRM showed even greater resilience, with optimal θ values around 0.20-0.30. At 40% corruption, RRM achieved accuracies significantly higher than ERM, with improvements up to 14.6% at 30% corruption.
  • AIRBUS Dataset (with Adam optimizer): RRM consistently delivered better or equivalent accuracy than ERM across all corruption levels. A θ value of 0.25 was found to be a strong performer.
  • AIRBUS Dataset (with SGD optimizer): RRM again demonstrated superior performance, particularly at higher corruption levels (20-40%), with improvements up to 12.7% over ERM. The optimal θ value was found to be around 0.30.

In summary, RRM consistently achieved accuracy levels comparable to or better than ERM, especially in corrupted scenarios. The ADH-LP architecture with the SGD optimizer was highlighted as a particularly robust combination.

Limitations and Future Work

  • Computational Cost: The paper notes that RRM ADH-LP requires slightly longer processing times compared to traditional methods, although this is justified by the improved accuracy and robustness.
  • u-Optimization Potential: The authors suggest that the u-optimization process, which currently excludes a significant portion of mislabeled images, could be further improved by incorporating novel techniques. This could potentially lead to even greater model accuracy and resilience.
  • Exploring More Complex Architectures: While the focus was on RRM's effectiveness rather than novel CNN architectures, future work could explore integrating RRM with more advanced deep learning models.
  • Real-World Deployment: The paper's findings have direct implications for real-world maritime surveillance systems, suggesting RRM as a promising approach for enhancing their reliability in challenging, data-noisy environments.

Summary updated

Abstract

This study proposes developing a resilient machine learning algorithm based on neural networks to classify naval images used in surveillance, search, and detection operations in vast coastal and marine environments. Coastal areas critical for water resource management often face challenges such as illegal fishing, trafficking, piracy, and other illicit activities that require robust monitoring systems powered by computer vision. However, real-world datasets in such environments can be compromised by label noise due to random inaccuracies or deliberate adversarial attacks, leading to decreased accuracy in machine learning models. Our innovative approach employs Rockafellian Risk Minimization (RRM) to mitigate the impact of label noise contamination, crucial to maintaining data integrity in water-related security and governance operations. Unlike existing methodologies that rely on extensively cleaned datasets, our two-step process adjusts neural network weights and manipulates nominal probabilities of data points to isolate potential data corruption effectively. This technique reduces dependence on meticulous data cleaning, thereby increasing data processing efficiency in water resources and coastal management. To validate the effectiveness and reliability of the proposed model, we apply RRM in various parameter settings to datasets specific to naval environments and evaluate its classification accuracy against traditional methods. By leveraging the proposed model, we aim to reinforce the robustness of ship detection models, ultimately contributing to developing more reliable automated maritime surveillance systems. Such systems are essential for strengthening governance, security, and water management and curbing illegal activities at sea.
BibTeX
@inproceedings{rangel2025efficient,
  author = {Gabriel Custódio Rangel and Vander Alves and Igor Pınheıro de Araújo Costa and Miguel Ângelo Lellis Moreira and Arthur Pinheiro de Araújo Costa and Marcos dos Santos and Eric Eckstrand},
  title = {{Efficient Naval Surveillance: Addressing Label Noise with Rockafellian Risk Minimization for Water Security}},
  booktitle = {Water},
  year = {2025},
  doi = {10.3390/w17030401},
}
Figure