Optimization of the occlusion strategy in visual tracking

Jinghuan Wen, Huimin Ma, Xiaoqin ZhangTsinghua 2016 · DOI

Abstract

Interference and anti-interference are two opposite and important issues in visual tracking. Occlusion interference can disguise the features of a target and can also be used as an effective benchmark to determine whether a tracking algorithm is reliable. In this paper, we proposed an inner Particle Swarm Optimization (PSO) algorithm to locate the optimal occlusion strategy under different tracking conditions and to identify the most effective occlusion positions and direction of movement to allow a target to evade tracking. This algorithm improved the standard PSO process in three ways. First, it introduced a death process, which greatly reduced the time cost of optimization. Second, it used statistical data to determine the fitness value of the particles so that the fitness more accurately described the tracking. Third, the algorithm could avoid being trapped in local optima, as the fitness changes with time. Experimental results showed that this algorithm was able to identify a global optimal occlusion strategy that can disturb the tracking machine with 86.8% probability over more than 10000 tracking processes. In addition, it reduced the time cost by approximately 80%, compared with conventional PSO algorithms.

Introduction: The Evasion Perspective

In computer vision research, the primary focus is typically on enhancing the robustness of tracking algorithms. However, understanding how a target can effectively evade a tracker is equally critical for developing truly resilient systems. This paper explores dynamic occlusion—specifically using interference like smoke or moving obstacles—to disguise a target's features.

Finding an optimal evasion strategy is computationally intensive due to high simulation costs (each run takes approximately 30 seconds) and the binary nature of the feedback (success or failure). Traditional optimization methods often struggle with this "all-or-nothing" data landscape and frequently get trapped in local optima.

Methodology: The Inner PSO Algorithm

To address these challenges, we propose an "inner PSO" algorithm tailored for the constraints of visual tracking simulations. The framework introduces three primary innovations:

1. The Death Process

To mitigate the high time cost, the algorithm calculates the distance of each particle from the current best-known position. Particles that are too distant—effectively "starving" for relevant data—are removed from the search loop. This culling process reduces the swarm to the most promising 20%, resulting in a roughly five-fold increase in speed.

2. Statistical Fitness Function

Because raw tracking data is binary (1 for success, 0 for failure), traditional gradient-based optimization is difficult. We developed a fitness function that considers the results of neighboring particles. By weighting successful occlusions and penalizing failures within a local neighborhood, the algorithm transforms a jagged binary landscape into a smooth, continuous surface that can be navigated effectively.

3. Dynamic Local Optima Avoidance

The fitness function is designed to evolve over time based on the statistical distribution of the swarm. This prevents particles from clustering permanently around sub-optimal points, allowing the swarm to "leap" to superior evasion strategies as they are discovered.

Experimental Setup and Simulation

The study utilized a custom-built Virtual Smoke Occlusion Simulation System. This environment models natural elements such as wind speed, direction, and temperature to create a realistic infrared-based tracking scenario.

The Virtual Smoke Occlusion Simulation System, showing the tracking interface (top left), radar positioning (top right), and the 3D environment (bottom).

Over 15,000 simulations were performed across seven different tracking conditions, varying approach angles and wind vectors, to validate the algorithm's effectiveness.

Key Results and Performance

The optimized algorithm demonstrated significant improvements over conventional methods:

  • Efficiency: Computation time was reduced by approximately 80%.
  • Effectiveness: The algorithm identified global optimal occlusion strategies that successfully disturbed tracking machines with an 86.8% probability across more than 10,000 test cases.
  • Tracker Vulnerability: The system was tested against various tracking styles, including adaptive window, fasten window, and group window. It revealed that while adaptive window trackers (often based on mean-shift) are generally robust, they remain vulnerable to precisely optimized smoke placement.
Sequence of optimized occlusion (a-d) demonstrating how smoke placement successfully shifts the tracking window away from the target ship.

Conclusion

By optimizing the occlusion strategy, we have created a powerful benchmark for visual tracking. This "hacker" approach allows developers to actively attack their own algorithms to find specific weaknesses. While currently limited to simulation, future work will focus on bridging the "sim-to-real" gap, accounting for complex fluid dynamics and real-world atmospheric interference.

Summary updated

Abstract

Interference and anti-interference are two opposite and important issues in visual tracking. Occlusion interference can disguise the features of a target and can also be used as an effective benchmark to determine whether a tracking algorithm is reliable. In this paper, we proposed an inner Particle Swarm Optimization (PSO) algorithm to locate the optimal occlusion strategy under different tracking conditions and to identify the most effective occlusion positions and direction of movement to allow a target to evade tracking. This algorithm improved the standard PSO process in three ways. First, it introduced a death process, which greatly reduced the time cost of optimization. Second, it used statistical data to determine the fitness value of the particles so that the fitness more accurately described the tracking. Third, the algorithm could avoid being trapped in local optima, as the fitness changes with time. Experimental results showed that this algorithm was able to identify a global optimal occlusion strategy that can disturb the tracking machine with 86.8% probability over more than 10000 tracking processes. In addition, it reduced the time cost by approximately 80%, compared with conventional PSO algorithms.
BibTeX
@inproceedings{wen2016optimization,
  author = {Jinghuan Wen and Huimin Ma and Xiaoqin Zhang},
  title = {{Optimization of the occlusion strategy in visual tracking}},
  booktitle = {Tsinghua},
  year = {2016},
  doi = {10.1109/tst.2016.7442504},
}
Figure