Generating Sea Surface Object Image Using Image-to-Image Translation

Wenbin Yin, Jun Yu, Zhiyi HuInternational Journal of Advanced Network Monitoring and Controls 2021 · DOI

Abstract

Training conditional adversarial networks for sea object detection requires a large number of images to solve image-to-image translation problems. In cases of insufficient samples, networks often suffer from overfitting and poor training results. This project proposes a conditional adversarial generative model that retains original background features even in the absence of paired samples. The primary goal is to reduce the deviation of the output from the original input. The approach involves labeling object images of different categories with color masks and then generating sea objects randomly within the original background. Experimental results demonstrate that, compared to other conditional adversarial generative models, this approach produces object images with richer textures and clearer structures.

Introduction

In maritime domains such as search and rescue or military reconnaissance, training robust object detection models is hindered by the scarcity of high-quality data. Acquiring images of ships at sea is expensive, often restricted by confidentiality, and technically challenging due to environmental factors like reflections and haze. While Generative Adversarial Networks (GANs) offer a solution for synthesizing data, they typically require massive training volumes to avoid overfitting—where the model memorizes data rather than learning to generate realistic variations. Furthermore, standard image-to-image translation often loses fine-grained background details. This research aims to generate realistic ships in specific locations on a sea background while maintaining the structural integrity of both the object and the environment.

Methodology

Architecture

The proposed model builds upon the pix2pix framework but introduces a specialized Encoder-Converter-Decoder architecture. Unlike standard models that use a simple encoder-decoder bottleneck, this architecture incorporates a "Converter" module consisting of six ResNet (Residual Network) blocks. This intermediate stage is critical for retaining original background features and spatial consistency that are typically lost during deep convolution operations.

The proposed Generator architecture featuring Encoding, Transformation (ResNet blocks), and Decoding stages.

Conditional Generative Framework

The system utilizes a Conditional Generative Adversarial Network (CGAN) framework. It takes a color mask—a simplified label where specific RGB values represent different object categories—and overlays it onto a sea background. This allows for precise control over the type and placement of the generated ship.

The overall Conditional GAN framework illustrating the interaction between the Generator (G) and Discriminator (D).

Discriminator and Loss Functions

To encourage sharper local textures, the model employs a Patch-GAN Discriminator. Rather than evaluating the entire image, it judges $70 \times 70$ blocks individually to determine authenticity. The optimization process uses a combination of Least Squares GAN loss for stability and L1 distance to reduce blurring, forcing the output to remain structurally close to the ground truth.

Dataset and Preprocessing

The study utilized a pool of 42,556 images from Kaggle and Google Earth. Rigorous preprocessing was applied to ensure quality:

  • Filtering: Manual selection of clear images and removal of heavy reflections via median filtering.
  • Annotation: Objects were labeled with specific RGB tags (e.g., (100,0,0) for ships).
  • Augmentation: Random rotations, horizontal flips, and brightness adjustments were used to expand the dataset.
  • Final Set: The refined dataset consisted of 1,334 images, split into 1,067 for training and 267 for verification.

Results and Discussion

The proposed model was benchmarked against Pix2pixGAN, Bicycle-GAN, and Pix2pix-HD.

Quantitative Performance: * PSNR (Peak Signal-to-Noise Ratio): The model achieved 18.453, outperforming competitors (ranging from 16.8 to 18.0), indicating higher fidelity. * SSIM (Structural Similarity Index): The model reached 90.39%, significantly higher than the next best model (Bicycle-GAN at 87.61%), suggesting superior structural consistency.

Qualitative Findings: Visually, the inclusion of the ResNet-based "Converter" proved essential. While other models suffered from texture distortion and background blurring, this approach produced ships with richer textures and clearer structural edges.

Conclusion

By integrating a ResNet-based transformation layer into a conditional GAN framework, this research successfully generates high-quality synthetic maritime imagery. This method effectively bridges the gap between abstract feature extraction and realistic image synthesis, providing a viable path for generating training data in data-scarce environments. Future work will focus on scaling the architecture for high-definition imagery and testing performance in diverse open-ocean weather conditions.

Summary updated

Abstract

Abstract Sea objects training, the conditional adversarial networks require a large number of images to solve image-to-image translation problems. In the case of insufficient samples, it leads to network overfitting and poor training results. This project proposes a conditional adversarial generative model that retains the original background features in the absence of paired samples. The goal of this project is to reduce the deviation of the corresponding output from the original input. Firstly, the object images of different categories are labeled with color masks. Second, sea objects are generated randomly in the original background using model of this project. Finally, the generated results of this approach are compared with other approaches. The experimental results show that, compared with results from other conditional adversarial generative models, the generated object images using model of this project have the characteristics of richer texture and clearer structure.
BibTeX
@article{yin2021generating,
  author = {Wenbin Yin and Jun Yu and Zhiyi Hu},
  title = {{Generating Sea Surface Object Image Using Image-to-Image Translation}},
  journal = {International Journal of Advanced Network Monitoring and Controls},
  year = {2021},
  doi = {10.21307/ijanmc-2021-016},
}
Figure