Abstract
Infrared (IR) sensors are critical for night-time surveillance but face significant challenges during thermal crossover events, where target and background temperatures converge. This paper proposes an online non-negative multi-modality feature template learning framework that integrates RGB data to enhance IR tracking. By enforcing non-negative constraints, the model learns robust part-based representations. An adaptive weighting mechanism dynamically prioritizes the most reliable modality, while Huber loss is utilized to mitigate the impact of contaminated data. Evaluations demonstrate that the proposed method achieves a success rate of 0.9, substantially outperforming existing state-of-the-art trackers.
Introduction
Infrared imaging is a staple of modern surveillance due to its ability to detect thermal signatures in low-light conditions. However, IR tracking is often compromised by thermal crossover—a phenomenon where the target's heat signature matches the background, causing the object to become indistinguishable. While multi-modality systems (RGB-IR) offer a solution, simple data fusion often fails in the presence of occlusions, cluttered backgrounds, and sensor noise. There is a critical need for a tracking framework that can intelligently weigh sensor reliability and learn robust object features in real-time.
Methodology
The proposed framework addresses these challenges through three core innovations:
Non-Negative Part-Based Learning
Unlike standard models that use abstract global features, this method applies Non-Negative Matrix Factorization (NMF) logic. By enforcing non-negative constraints, the tracker is forced to represent objects as a "sum of parts" (e.g., head, torso, limbs). This part-based approach ensures that if one part of the target is occluded, the remaining visible parts can maintain a stable track.
Adaptive Modality Weighting
The system does not treat RGB and IR data as equally reliable. Instead, it calculates a dynamic importance weight ($\alpha$) for each modality. When the IR signal degrades—such as during thermal crossover—the model automatically shifts its trust toward the RGB spectrum, ensuring continuous tracking.
Robust Decontamination with Huber Loss
To prevent the model from "learning" background noise or occlusions, the authors utilize the Huber loss function. While an explicit error term ($E$) was initially considered for noise handling, the Huber loss formulation provided equivalent "decontamination" with significantly lower computational overhead, streamlining the optimization process.
Experimental Results
The framework was evaluated on 15 aligned RGB-IR video pairs featuring challenges like scale variation, poor illumination, and thermal crossover. The proposed method demonstrated superior performance across key metrics:
- Overlapping Rate: Achieved a score of 0.7, compared to 0.58 for the next best competitor (L1).
- Success Rate: Reached 0.9, a significant improvement over the 0.69 achieved by state-of-the-art multi-modality trackers.
Qualitative analysis shows that while competing trackers often drift toward the background during crossover events, the proposed model maintains a precise bounding box by leveraging its adaptive weighting and part-based templates.
Conclusion
This research introduces a robust multi-modality tracking framework that effectively mitigates the limitations of infrared sensors. By combining non-negative part-based learning with adaptive sensor weighting, the system achieves high accuracy in complex environments. Future work will focus on optimizing the iterative algorithms to achieve real-time processing speeds through hardware acceleration and algorithmic streamlining.
Summary updated