Enhancing Robustness of Human Detection Algorithms in Maritime SAR through Augmented Aerial Images to Simulate Weather Conditions

Miguel Tjia, Artem Kim, Elaine Wynette Wijaya, Hanna Tefara, Kevin ZhuarXiv 2024 · DOI · arXiv

Introduction

Maritime Search and Rescue (SAR) operations are inherently time-critical. Statistical data suggests that survival rates for individuals lost at sea drop exponentially after the first 18 hours, making rapid detection a matter of life and death. While Unmanned Aerial Vehicles (UAVs) have become essential tools for scanning vast oceanic expanses, the computer vision models that power them often fail in the field. Most state-of-the-art object detection models, such as the YOLO (You Only Look Once) family, are trained on high-quality, "fair weather" datasets. When these models encounter the reality of maritime environments—characterized by heavy rain, dense fog, and low-light conditions—their detection accuracy often plummets. This research addresses this gap by developing a robust training methodology using synthetic weather augmentation.

Methodology and Model Selection

The study evaluated four specific iterations of the YOLO architecture: YOLOv5s, YOLOv5l, YOLOv10s, and YOLOv10l. The selection of both "small" (s) and "large" (l) variants allowed researchers to analyze the trade-off between real-time processing speed (latency) and detection accuracy.

A significant technical focus was placed on YOLOv10, which introduces "NMS-free" (Non-Maximum Suppression) training. By eliminating the need for redundant bounding box removal during inference, YOLOv10 aims to reduce latency, a critical factor for edge deployment on drones. Throughout the testing phase, the primary metric of success was recall. In SAR contexts, a false positive (identifying debris as a human) results in a minor delay, whereas a false negative (missing a person) is catastrophic. Thus, the models were optimized to ensure that if a human is present in the frame, they are flagged regardless of environmental noise.

Dataset and Synthetic Augmentation

The research utilized a combined dataset of 13,282 raw aerial images sourced from the Aerial dataset of Floating Objects (AFO) and the Human Swimming/Floating Dataset. To simulate the unpredictable nature of the open sea, the team expanded this dataset to 26,548 images through digital transformations.

Figure 1: Overview of synthetic weather augmentations applied to the original maritime imagery, including night, rain, and fog simulations.

As shown in the figure above, the original clear-day images (a) were transformed into six distinct categories:

  • Fog: Simulated by blending images with a semi-transparent white layer.
  • Sunny: Enhanced via increased brightness and yellow tinting.
  • Rain: Created by overlaying specific rain-streak textures.
  • Night: Achieved by reducing contrast/brightness and applying a blue tint.

The final training set was balanced with 50% clear-day images and 50% distributed across these adverse weather simulations, forcing the neural networks to learn structural human features that are independent of ideal lighting.

Key Results and Performance Gains

The inclusion of augmented weather data led to measurable improvements in model robustness. The YOLOv5l model emerged as the top performer, achieving a human recall rate of 0.911 (a 3.4% improvement over the baseline) and a high F1-score of 0.94. This indicates that the model is exceptionally reliable at distinguishing humans from inanimate objects like kayaks or floating debris, even in low-visibility scenarios.

Figure 2: Example of a Night-Rainy simulation, demonstrating the extreme low-visibility conditions the models were trained to navigate.

Interestingly, while YOLOv10 is the more recent architecture, the larger YOLOv5l variant provided the best balance of precision and recall in this specific maritime context. However, YOLOv10l also saw a significant boost, with recall rising from 0.87 to 0.91 when trained on the augmented dataset.

Discussion and Future Directions

While synthetic transformations proved highly effective for boosting model recall, the authors acknowledge limitations in the current approach. Digital filters are "flat" and do not fully capture the complex physics of light scattering in real fog or the way rain droplets interact with camera lenses.

To further enhance SAR capabilities, future work will explore the use of Generative Adversarial Networks (GANs), specifically CycleGANs, to create more naturalistic weather transitions. Additionally, validating these models against real-world footage captured during actual maritime storms remains the ultimate goal for ensuring these AI systems are ready for deployment in life-saving missions.

Summary updated

Abstract

7,651 cases of Search and Rescue Missions (SAR) were reported by the United States Coast Guard in 2024, with over 1322 SAR helicopters deployed in the 6 first months alone. Through the utilizations of YOLO, we were able to run different weather conditions and lighting from our augmented dataset for training. YOLO then utilizes CNNs to apply a series of convolutions and pooling layers to the input image, where the convolution layers are able to extract the main features of the image. Through this, our YOLO model is able to learn to differentiate different objects which may considerably improve its accuracy, possibly enhancing the efficiency of SAR operations through enhanced detection accuracy. This paper aims to improve the model's accuracy of human detection in maritime SAR by evaluating a robust datasets containing various elevations and geological locations, as well as through data augmentation which simulates different weather and lighting. We observed that models trained on augmented datasets outperformed their non-augmented counterparts in which the human recall scores ranged from 0.891 to 0.911 with an improvement rate of 3.4\% on the YOLOv5l model. Results showed that these models demonstrate greater robustness to real-world conditions in varying of weather, brightness, tint, and contrast.
BibTeX
@misc{tjia2024enhancing,
  author = {Miguel Tjia and Artem Kim and Elaine Wynette Wijaya and Hanna Tefara and Kevin Zhu},
  title = {{Enhancing Robustness of Human Detection Algorithms in Maritime SAR through Augmented Aerial Images to Simulate Weather Conditions}},
  year = {2024},
  eprint = {2408.13766},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.13766},
}
Figure