Searching a Lightweight Network Architecture for Thermal Infrared Pedestrian Tracking

Peng Gao, Xiao Liu, Yu Wang, Ru-Yue YuanarXiv 2024 · DOI · arXiv

Introduction

Thermal Infrared Pedestrian Tracking (TIR-PT) is a critical component for autonomous systems operating in challenging environments, such as low-light or foggy conditions where standard RGB sensors often fail. Despite its importance, many current TIR trackers utilize "off-the-shelf" backbones like AlexNet or ResNet, which were originally optimized for color image classification. These heavy-duty models are often redundant for thermal data, which typically lacks texture and exhibits lower contrast. To address this, researchers have developed an automated method to design lightweight neural networks specifically tailored for TIR pedestrian tracking.

Methodology

The proposed approach follows a two-stage pipeline: Search and Retrain. This framework allows for the discovery of an optimal architecture that balances tracking accuracy with computational efficiency.

The two-stage framework: Stage 1 involves searching for operation and network architecture candidates, while Stage 2 focuses on retraining the selected model with joint supervision.

Neural Architecture Search (NAS)

The search phase utilizes a differentiable NAS strategy inspired by DARTS. By relaxing the search space from discrete choices to continuous ones, the model can learn its own optimal configuration. To manage the high memory requirements typically associated with NAS, the authors implemented a Random Channel Selection strategy. By feeding only a random 1/4 of the feature channels into operation candidates during search, the process was accelerated from 3.3 GPU days to just 1.9 without compromising performance.

Dual-Bottom Cell Design

A key innovation in the search space is the use of Dual-bottom cells. Unlike single-bottom cells that only receive input from the immediate predecessor, dual-bottom cells pull information from two previous layers (S0 and S1). This structure is particularly effective at preserving spatial information, which is vital for tracking small, low-contrast targets in thermal imagery.

The internal architecture of a searched cell, illustrating the multi-path connections from input states S0 and S1 to the final concatenated output.

Joint Supervision and Retraining

Once the optimal architecture is identified, it is retrained using a triple-threat loss function consisting of Classification Loss to distinguish pedestrians from the background, Batch Hard Triplet Loss to enhance identity preservation, and Center Loss to ensure features of the same target cluster tightly in the feature space.

Experimental Results

The model was evaluated on two major benchmarks: LSOTB-TIR (over 600k frames) and PTB-TIR (30k frames). The authors notably demonstrated that the architecture searched on LSOTB-TIR could be directly transferred to PTB-TIR with high performance, proving the model's robustness.

  • Performance: On LSOTB-TIR, the tracker achieved a 0.805 Precision and 0.669 Success rate, significantly outperforming the baseline DiMP (0.739 Precision). On PTB-TIR, it reached a 0.832 Precision, surpassing established methods like ECO-deep and MDNet.
  • Efficiency: The final searched model is remarkably lightweight. One configuration utilizes only 1.28M parameters and 0.39 GFLOPs, making it highly suitable for real-time deployment on edge devices.

Conclusion and Future Work

This research demonstrates that for specialized tasks like TIR tracking, the architecture's design is more critical than its depth. The searched lightweight model outperformed much larger ResNet-50 based trackers while remaining computationally efficient. Future work will focus on improving resilience against severe occlusion and visual distractors by integrating advanced attention mechanisms.

Summary updated

BibTeX
@misc{gao2024searching,
  author = {Peng Gao and Xiao Liu and Yu Wang and Ru-Yue Yuan},
  title = {{Searching a Lightweight Network Architecture for Thermal Infrared Pedestrian Tracking}},
  year = {2024},
  eprint = {2402.16570},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.16570},
}
Figure