TL;DR
This paper introduces a novel "Noise Adversarial Network" (NAN) to improve object detection in sonar images, which are notoriously noisy and dataset-scarce. By training a detector with adversarial perturbations mimicking sonar noise, the method significantly boosts performance and robustness, even showing promise on optical image datasets.
What Problem Does This Paper Address?
Object detection in sonar images is crucial for underwater tasks like mine detection and fish tracking. However, it faces two major hurdles: a scarcity of diverse sonar image datasets and the pervasive presence of various noise types (like reverberation) that degrade image quality. Standard object detectors, trained primarily on clean optical images, struggle with these challenges, leading to overfitting and poor generalization to real-world noisy sonar data. The inherent differences between optical and sonar imaging further complicate direct application of existing methods.
Methods
The core innovation is the Noise Adversarial Network (NAN), a sideway network integrated into a standard object detection framework (Faster R-CNN in this case). Instead of adding noise directly to input images, NAN generates adversarial perturbations in the convolutional feature space. This approach is computationally efficient and allows for end-to-end training.
Key aspects of the method include:
- Adversarial Perturbation Generation: NAN learns to generate noise that mimics the statistical properties of sonar reverberation, specifically modeling its amplitude distribution as Rayleigh. It predicts the variance of this noise from feature maps.
- Noise Block (NB): An auxiliary component, NB, is introduced in earlier layers of the detector. It injects a basic form of speckle noise to provide prior knowledge to NAN, helping it estimate noise magnitudes more effectively.
- Stage-wise Training: The training process is split into two stages. First, the base detector is trained without NAN. Then, both the detector and NAN are jointly trained, allowing NAN to generate adversarial examples that guide the detector towards better generalization and noise robustness.
- Loss Function: The total loss combines the standard detection losses (classification and bounding box regression) with an adversarial loss that encourages the detector to approximate the true distribution of data, making it robust to the generated noise.
Datasets
- Custom Sonar Image Dataset: The authors created a new dataset of 216 sonar images captured by Side Scan Sonar (SSS) and Synthetic Aperture Sonar (SAS). This dataset features a variety of object sizes, illumination, positions, and noise distributions, with three categories: corpse, shipwreck, and plane wreckage. It was augmented to create training (A+, 4200 samples) and testing (B+, 1200 samples) subsets.
- PASCAL VOC 2007: Used to demonstrate the universality of the proposed method on optical images.
Key Insights and Ideas
- Feature Space Perturbation is Key: Generating adversarial noise in the feature space, rather than pixel space, is more efficient and effective for sonar image detection.
- Mimicking Sonar Noise Properties: Explicitly modeling the Rayleigh distribution of sonar reverberation amplitude is crucial for generating realistic and effective adversarial examples.
- Auxiliary Noise Block (NB) Enhances NAN: The Noise Block, by providing early-stage noise information, significantly improves NAN's ability to generate robust perturbations and boosts overall detector performance.
- Generalization Beyond Sonar: The method shows promise not only for noisy sonar images but also for improving robustness on optical images when subjected to similar noise types.
Key Results
- Significant mAP Boost on Sonar Data: The proposed method, when integrated with Faster R-CNN (ResNet-101 backbone), achieved an 8.9% mAP boost on their custom sonar dataset compared to the baseline.
- Improved Noise Robustness: The detector equipped with NAN and NB demonstrated significantly better performance under various intensities of speckle noise attack, with mAP increasing from 5.9% to 79.6% at maximum noise variance (σ²=1) on VGG16.
- Enhanced Generalization: The method showed improved performance on the PASCAL VOC 2007 optical dataset, achieving a 2.4% mAP boost with Gaussian noise.
- Error Analysis: NAN and NB effectively reduced localization errors (Loc) and confusion with similar objects (Sim), decreasing them from 44% to 8% in top-ranked detections under noise attack.
Limitations and Future Work
- Limitations of Noise Augmentation: While Noise Augmentation (NA) helps, combining it with NAN and NB sometimes led to a decrease in performance, possibly due to excessive noise addition.
- Specific Noise Model: The current model focuses on reverberation noise. The authors suggest exploring other inherent properties like illumination and shadow for further performance improvements.
- Future Work: The authors plan to investigate other inherent properties of sonar images, such as illumination and shadow, to further enhance detector performance. They also aim to release their custom sonar image dataset to facilitate future research.
Summary updated