Miss Detection vs. False Alarm: Adversarial Learning for Small Object Segmentation in Infrared Images

Huan Wang, Luping Zhou, Lei WangICCV 2019 · OpenReview

Abstract

A key challenge of infrared small object segmentation (ISOS) is to balance miss detection (MD) and false alarm (FA). This usually needs "opposite" strategies to suppress the two terms, and has not been well resolved in the literature. In this paper, we propose a deep adversarial learning framework to improve this situation. Departing from the tradition of jointly reducing MD and FA via a single objective, we decompose this difficult task into two sub-tasks handled by two models trained adversarially, with each focusing on reducing either MD or FA. Such a new design brings forth at least three advantages. First, as each model focuses on a relatively simpler sub-task, the overall difficulty of ISOS is somehow decreased. Second, the adversarial training of the two models naturally produces a delicate balance of MD and FA, and low rates for both MD and FA could be achieved at Nash equilibrium. Third, this MD-FA detachment gives us more flexibility to develop specific models dedicated to each sub-task. To realize the above design, we propose a conditional Generative Adversarial Network comprising of two generators and one discriminator. Each generator strives for one sub-task, while the discriminator differentiates the three segmentation results from the two generators and the ground truth. Moreover, in order to better serve the sub-tasks, the two generators, based on context aggregation networks, utilize different size of receptive fields, providing both local and global views of objects for segmentation. As verified on multiple infrared image data sets, our method consistently achieves better segmentation than many state-of-the-art ISOS methods.

Introduction

In infrared imagery, objects are often just a few pixels wide, extremely dim, and buried in cluttered backgrounds like clouds or heavy terrain. Segmenting these "needles in a haystack" presents a fundamental conflict between Miss Detection (MD) and False Alarm (FA). To avoid missing a target, a model needs high sensitivity (low threshold), but to avoid mistaking noise for a target, it needs to be conservative (high threshold). Traditional methods attempt to optimize a single objective to handle both, often resulting in a mediocre compromise. Furthermore, standard deep learning architectures often fail because their pooling layers can accidentally "erase" single-pixel targets.

The MDvsFA-cGAN Framework

To address this, we propose a conditional Generative Adversarial Network (cGAN) with a unique "divide and conquer" architecture. Instead of a single model, we employ two specialized generators and a shared discriminator.

The MDvsFA-cGAN architecture featuring two specialized generators ($G_1$ and $G_2$) and a multi-input discriminator.

Dual Generators ($G_1$ and $G_2$)

  • $G_1$ (The Hunter): Designed to minimize MD. It utilizes a smaller receptive field to focus on local visual features, ensuring tiny footprints are not overlooked.
  • $G_2$ (The Skeptic): Designed to minimize FA. It employs a much larger receptive field and skip connections to gain a global view, helping it distinguish actual targets from background clutter like cloud edges.

The Discriminator ($D$)

The discriminator's role is to distinguish between the outputs of $G_1$, $G_2$, and the actual ground truth. This competition forces both generators to produce results that are not just specialized, but also realistic. To prevent the two generators from drifting too far apart, a Generator Consistency Loss is applied, measuring the difference between their feature maps within the discriminator.

Training Dynamics and Nash Equilibrium

During training, we apply an Asymmetric Data Loss. $G_1$ is penalized more heavily for missing pixels, while $G_2$ is penalized more for false positives. In a standard model, one must manually tune a threshold to balance MD and FA. In this adversarial setup, the competition between the generators naturally finds a "delicate balance" at the Nash equilibrium without manual intervention.

Visualization of the training process showing the evolution of $G_1$ and $G_2$ outputs across epochs. $G_1$ initially produces more false alarms, which are gradually suppressed through adversarial training.

Experimental Results

The proposed method, MDvsFA-cGAN, was evaluated against 14 state-of-the-art ISOS methods across several datasets, including a newly curated collection of real infrared sequences and synthetic data.

  • Superior F-measure: On the "Single" dataset, it achieved an F-measure of 0.60, significantly outperforming the next best deep learning model (scGAN at 0.50) and traditional ISOS methods.
  • Precision/Recall Balance: While some models had higher recall, they suffered from very low precision. MDvsFA-cGAN maintained the best balance, achieving the highest precision (0.66) on the "Single" dataset.
  • Robustness: Ablation studies showed that the model is relatively insensitive to specific hyperparameter weights, suggesting the adversarial framework effectively manages the MD-FA trade-off.

Conclusion

By detaching the MD and FA objectives into two specialized models, the overall difficulty of the ISOS task is reduced. The use of asymmetric receptive fields allows the system to leverage both local and global context, resulting in a robust segmentation framework that outperforms existing state-of-the-art approaches. Future work will explore incorporating temporal context from video sequences to further distinguish moving targets from static background clutter.

Summary updated

Abstract

A key challenge of infrared small object segmentation (ISOS) is to balance miss detection (MD) and false alarm (FA). This usually needs ``opposite'' strategies to suppress the two terms, and has not been well resolved in the literature. In this paper, we propose a deep adversarial learning framework to improve this situation. Departing from the tradition of jointly reducing MD and FA via a single objective, we decompose this difficult task into two sub-tasks handled by two models trained adversarially, with each focusing on reducing either MD or FA. Such a new design brings forth at least three advantages. First, as each model focuses on a relatively simpler sub-task, the overall difficulty of ISOS is somehow decreased. Second, the adversarial training of the two models naturally produces a delicate balance of MD and FA, and low rates for both MD and FA could be achieved at Nash equilibrium. Third, this MD-FA detachment gives us more flexibility to develop specific models dedicated to each sub-task. To realize the above design, we propose a conditional Generative Adversarial Network comprising of two generators and one discriminator. Each generator strives for one sub-task, while the discriminator differentiates the three segmentation results from the two generators and the ground truth. Moreover, in order to better serve the sub-tasks, the two generators, based on context aggregation networks, utilzse different size of receptive fields, providing both local and global views of objects for segmentation. As verified on multiple infrared image data sets, our method consistently achieves better segmentation than many state-of-the-art ISOS methods.
BibTeX
@inproceedings{wang2019miss,
  author = {Huan Wang and Luping Zhou and Lei Wang},
  title = {{Miss Detection vs. False Alarm: Adversarial Learning for Small Object Segmentation in Infrared Images}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  url = {https://openreview.net/forum?id=oJ821yB-d69},
}
Figure