PixelGame: Infrared Small Target Segmentation as a Nash Equilibrium

Heng Zhou, Chunna Tian, Zhenxi Zhang, Chengyang Li, Yongqiang Xie, Zhongbo Li

Abstract

Infrared Small Target Segmentation (ISTS) is a critical task in computer vision, characterized by the difficulty of detecting dim, sparse targets against complex, noisy backgrounds. Traditional methods often struggle to balance the antagonistic relationship between False Negatives (FNs) and False Positives (FPs). This paper proposes PixelGame, a novel framework that reformulates ISTS as a competitive game between two neural network players. By reaching a Nash equilibrium, the model achieves an optimal trade-off between detection sensitivity and precision. To support this framework, we introduce a Fully Dilated Convolution Network (FDCN) to preserve spatial resolution and a Maximum Information Modulation (MIM) module to highlight local extreme points. Experimental results on the NUST-ISTS and NUAA-ISTS datasets demonstrate that PixelGame significantly outperforms state-of-the-art methods.

1. Introduction

In infrared imaging, small targets often occupy only a few pixels and lack distinct shape or texture. They are frequently buried in heavy background clutter, such as clouds, sea-sky lines, or urban structures. The fundamental challenge in ISTS is the "tug-of-war" between sensitivity and precision. Increasing sensitivity to capture dim targets often leads to a surge in false alarms, while tightening precision constraints causes the model to miss faint signals.

Most existing deep learning approaches attempt to resolve this conflict using a single objective function, typically a weighted sum of different loss terms. However, finding the optimal weights is a labor-intensive process that rarely generalizes across diverse environments. PixelGame addresses this by treating the segmentation task as a multi-player strategy game, where the optimal solution emerges from competition rather than manual compromise.

2. Methodology

2.1 The PixelGame Framework

PixelGame decomposes the segmentation task into a strategic interaction between two sub-networks, or "players":

  • The FNs-player ($S_1$): This player is incentivized to minimize False Negatives. It adopts an aggressive strategy, attempting to identify every potential target pixel, even at the risk of including noise.
  • The FPs-player ($S_2$): This player is incentivized to minimize False Positives. It adopts a conservative strategy, focusing only on pixels with high confidence to ensure precision.

The interaction is governed by a utility function that penalizes both missing targets and false alarms. Through iterative training, the players seek a Nash equilibrium—a state in which neither player can improve their utility by unilaterally changing their strategy. This equilibrium represents a mathematically grounded "sweet spot" for segmentation performance.

2.2 Architectural Innovations

To ensure the players have the necessary information to compete effectively, the authors introduce two key architectural components:

Fully Dilated Convolution Network (FDCN): Standard convolutional neural networks (CNNs) rely on downsampling (pooling) to increase the receptive field. In ISTS, this can be catastrophic, as a 3x3 pixel target can be completely erased after a few layers of pooling. FDCN utilizes dilated convolutions to expand the receptive field without reducing spatial resolution. This ensures that the fine-grained details of tiny targets are preserved throughout the network.

Maximum Information Modulation (MIM): Attention mechanisms are commonly used to focus on relevant features. However, standard attention often uses average pooling, which can dilute the signal of a single bright pixel. MIM is designed specifically for the "point-source" nature of infrared targets. It employs Global Max Pooling and Cross-Channel Max Pooling to highlight local extreme points (the brightest spots) while suppressing the surrounding background clutter.

3. Experiments and Results

3.1 Datasets and Metrics

The model was rigorously tested on two benchmark datasets:

  • NUST-ISTS: A large-scale dataset containing 10,000 training patches with both synthetic and real-world backgrounds.
  • NUAA-ISTS: A dataset of 427 real-world infrared images featuring diverse scenarios like clouds and buildings.

Performance was measured using the F1-measure and Intersection over Union (IoU), which provide a comprehensive view of the balance between precision and recall.

3.2 Performance Comparison

PixelGame was compared against 23 traditional algorithms and 3 state-of-the-art CNN-based methods. On the NUAA-ISTS dataset, PixelGame achieved an F1-measure of 0.86 and an IoU of 0.75, surpassing the previous leading method, ALCNet. Qualitative analysis showed that PixelGame is particularly effective at ignoring high-intensity background noise, such as the sharp edges of buildings or bright cloud formations, which frequently cause false positives in other models.

3.3 Ablation Study

Ablation experiments confirmed the necessity of each component. Removing the game-theoretic utility function led to a significant imbalance between FNs and FPs, while replacing MIM with standard attention modules resulted in a drop in detection accuracy for the dimmest targets.

4. Discussion and Conclusion

The success of PixelGame suggests that competitive modeling is a powerful alternative to traditional single-objective optimization for tasks with inherent contradictions. By allowing two specialized agents to compete, the system naturally finds a more robust balance than a human-tuned loss function.

While PixelGame offers superior accuracy, it comes with increased computational costs. The model currently operates at approximately 9 frames per second (FPS), which is slower than some "single-actor" models that reach 100 FPS. Future work will focus on optimizing the efficiency of the Nash equilibrium search and incorporating temporal information to further distinguish moving targets from static background noise.

In summary, PixelGame provides a robust, physics-informed framework for infrared small target segmentation, demonstrating that in the face of conflicting objectives, competition can lead to the most effective compromise.

Summary updated

BibTeX
@article{zhou2022pixelgame,
  author = {Heng Zhou and Chunna Tian and Zhenxi Zhang and Chengyang Li and Yongqiang Xie and Zhongbo Li},
  title = {{PixelGame: Infrared Small Target Segmentation as a Nash Equilibrium}},
  journal = {IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens.},
  year = {2022},
  doi = {10.1109/jstars.2022.3206062},
}
Figure