IRSDT: A Framework for Infrared Small Target Tracking with Enhanced Detection

Jun Fan, Jingbiao Wei, Hai Huang, Dafeng Zhang, Ce Chen2023 · DOI

TL;DR

This paper introduces IRSDT, a novel framework for detecting and tracking small infrared vehicle targets in complex backgrounds. By combining lightweight CNNs for detection with a traditional tracker and a trajectory predictor, IRSDT achieves robust tracking even during temporary target disappearances and interference, outperforming existing methods with a low Euclidean distance deviation of ≤4 pixels.

What Problem Does This Paper Address?

Detecting and tracking small targets in infrared (IR) imagery is a persistent challenge. These targets often suffer from low resolution, lack distinct shape and texture features, and can be easily confused with background elements. This is particularly problematic in applications like forest fire prevention, field search and rescue, and night reconnaissance where reliable identification of small, distant objects is crucial. Existing methods, while improving, often struggle with computational costs, making them unsuitable for edge computing, and can be sensitive to factors like temporary occlusions, target deformation, and interference from similar objects.

Methods

The IRSDT framework is a three-stage pipeline:

  1. Full-Image Object Detection (yolo_IRS_1): A lightweight CNN-based model designed to improve the recall rate for small vehicle targets without significantly increasing computational cost. It utilizes a CSP backbone and a novel ARF module with dilated convolutions to capture multi-scale information effectively.
  2. Cropped-Image Object Detection and Tracking: This stage employs a two-branch approach.
  3. RoI Target Detection (yolo_IRS_2): A more precise, lightweight CNN model specifically designed for detecting small targets within a Region of Interest (RoI) cropped from the full image.
  4. RoI Target Tracking (KCF): The traditional Kernelized Correlation Filter (KCF) algorithm is used to track the target within the RoI. KCF is chosen for its adaptability to target deformation and occlusions, and its balance between computational cost and tracking stability.
  5. Target Trajectory Prediction: An inter-frame prediction method is introduced to estimate the target's coordinates when detection or tracking fails (e.g., during temporary disappearances). This involves a combination of a moving average acceleration (MAa) method and an optimized Kalman filter, with a warm-up mechanism to ensure smooth transitions.

Datasets

The study utilized a public dataset of infrared small vehicle targets under complex backgrounds [30]. The dataset contained 21,750 images and 89,174 targets, comprising 87 image sequences and 393 trajectories. For training and testing, images were resized to 640x640 pixels. The dataset was split into an odd-numbered group (87 sets, 11,000 images, 44,131 samples) for training and an even-numbered group (87 sets, 10,750 images, 45,043 samples) for testing. A specific RoI dataset (64x64 pixels) was created from this for the cropped-image detection and tracking experiments.

Key Insights and Ideas

  • Hierarchical Detection for Efficiency: Using a lightweight model (yolo_IRS_1) for initial full-image detection and then a more specialized model (yolo_IRS_2) on cropped RoIs significantly reduces computational load while maintaining detection accuracy.
  • Hybrid Tracking Approach: Combining a CNN-based detector with a traditional tracker (KCF) offers a robust solution. The CNN provides strong feature extraction, while KCF excels at handling motion and appearance changes within a localized region.
  • Predictive Tracking for Robustness: The trajectory prediction module is crucial for maintaining track continuity during temporary occlusions or when the target is obscured, a common failure point for purely detection-based trackers.
  • Adaptive Feature Fusion: The use of dilated convolutions in yolo_IRS_1's ARF module allows it to capture contextual information at different scales, which is vital for small targets that might otherwise be missed.

Key Results

  • The proposed IRSDT framework achieved an optimal recall rate and score in detection tasks, with a Euclidean distance deviation of ≤4 pixels for tracking.
  • Ablation studies showed significant improvements in precision and recall rates with the proposed backbone, neck, and detection head modifications compared to the Yolov5s baseline.
  • The yolo_IRS_2 model demonstrated higher precision (94.7%) and recall (92.5%) on the RoI dataset compared to Yolov5s (94.1% precision, 90.4% recall).
  • Experimental results on sequences showed the IRSDT framework could accurately identify and track vehicles, even when they disappeared behind buildings or trees, with tracking deviations generally within 2-4 pixels.

Limitations and Future Work

  • Computational Cost: While the framework aims to reduce computational overhead, the paper notes that the yolo_IRS_1 does not process every frame due to its higher computing costs. Further optimization for real-time processing of all frames might be beneficial.
  • Suboptimal Precision/Cost Trade-off: The paper acknowledges that while the model achieves an optimal recall rate and score, its precision rate and computational costs are suboptimal compared to some other methods. This suggests a potential trade-off that might be further explored.
  • Future Work: The authors suggest that their framework can accurately identify and track targets, performing well under interference and temporary disappearances. Future work could focus on further enhancing the detection and tracking capabilities in even more challenging scenarios, such as extreme weather conditions or dense clutter.

Summary updated

BibTeX
@misc{fan2023irsdt,
  author = {Jun Fan and Jingbiao Wei and Hai Huang and Dafeng Zhang and Ce Chen},
  title = {{IRSDT: A Framework for Infrared Small Target Tracking with Enhanced Detection}},
  year = {2023},
  doi = {10.3390/s23094240},
}
Figure