Introduction
This article presents formal corrections to the 2019 publication "SiamFT: An RGB-Infrared Fusion Tracking Method via Fully Convolutional Siamese Networks." The original paper established a robust framework for multi-modal object tracking; however, a clerical error led to the publication of an incorrect Figure 5. This update provides the corrected visualizations, which are essential for understanding how the SiamFT model dynamically balances visible and infrared data in challenging environments.
The Problem: Robust Multi-modal Tracking
Object tracking is a cornerstone of computer vision, but "visible-only" trackers often fail in real-world scenarios. Environmental factors such as extreme over-exposure (e.g., glare from headlights) or total darkness can render visible sensors ineffective. To solve this, researchers utilize RGB-Infrared (RGB-T) fusion. Visible cameras provide high-resolution texture and color, while infrared (thermal) cameras capture heat signatures regardless of lighting. The primary challenge is the fusion strategy: determining which sensor to trust at any given moment. SiamFT addresses this by adaptively weighting the contribution of each modality.
Methods: Adaptive Weighting in Siamese Networks
The SiamFT framework utilizes Fully Convolutional Siamese Networks to process parallel streams of visible (RGB) and infrared (IR) data. In a standard Siamese tracker, the network compares a template against a search region to find the best match. SiamFT extends this by calculating a "reliability" score for each modality, known as the modality response value.
Instead of a static 50/50 split, the weights are dynamic and change frame-by-frame based on the confidence of the tracking prediction in each spectrum. If the visible light response is weak or noisy (e.g., due to glare), its weight is reduced, and the system shifts reliance to the infrared feed.
Analysis of Corrected Visualizations
The corrected figures provide a clear demonstration of how multi-modal fusion handles sensor failure in two distinct types of challenging sequences.
Case 1: The "Hand-off" in Glare
In sequences involving extreme over-exposure, such as a car moving through intense light glare, the model demonstrates a seamless "hand-off" between sensors. As shown in Figure 1, when the car enters a region of extreme glare (around frame #70), the visible modality weight (blue line) crashes toward zero. Simultaneously, the infrared weight (red dashed line) spikes to nearly 1.0, allowing the tracker to maintain its target using thermal data alone.
Case 2: Consistent Thermal Dominance in Darkness
In low-light environments where visible features are nearly invisible, the model correctly identifies the infrared sensor as the primary source of truth. As illustrated in Figure 2, the infrared modality (red dashed line) remains the primary source of truth as the visible modality (blue line) struggles with low contrast. The infrared weight stays consistently above 0.8, proving the model's ability to adaptively predict which modality is more trustworthy.
Conclusion
The quantitative weight curves in the corrected figures demonstrate the model's sensitivity and robustness. Modality weights are not binary; they fluctuate between 0 and 1, allowing for a "soft" fusion that blends the best of both worlds when both sensors are partially reliable. These corrections confirm the effectiveness of the SiamFT adaptive weighting mechanism in mitigating sensor-specific failures in real-world tracking scenarios.
Summary updated