Abstract
Object detectors trained on large-scale RGB datasets are being extensively employed in real-world applications. However, these RGB-trained models suffer a performance drop under adverse illumination and lighting conditions. Infrared (IR) cameras are robust under such conditions and can be helpful in real-world applications. Though thermal cameras are widely used for military applications and increasingly for commercial applications, there is a lack of robust algorithms to robustly exploit the thermal imagery due to the limited availability of labeled thermal data. In this work, we aim to enhance the object detection performance in the thermal domain by leveraging the labeled visible domain data in an Unsupervised Domain Adaptation (UDA) setting. We propose an algorithm agnostic meta-learning framework to improve existing UDA methods instead of proposing a new UDA strategy. We achieve this by meta-learning the initial condition of the detector, which facilitates the adaptation process with fine updates without overfitting or getting stuck at local optima. However, meta-learning the initial condition for the detection scenario is computationally heavy due to long and intractable computation graphs. Therefore, we propose an online meta-learning paradigm which performs online updates resulting in a short and tractable computation graph. To this end, we demonstrate the superiority of our method over many baselines in the UDA setting, producing a state-of-the-art thermal detector for the KAIST and DSIAC datasets.
The Challenge: Bridging the Visible-to-Thermal Gap
Standard object detectors excel in daylight but often fail in the dark, through thick fog, or under blinding glare—conditions where visible-light cameras struggle. Thermal (Infrared) cameras are the perfect solution for these "adverse conditions," but they face a "data desert": there simply aren't enough labeled thermal datasets to train robust deep-learning models from scratch.
The common solution is Unsupervised Domain Adaptation (UDA), where a model trained on labeled RGB data (the source) is adapted to work on unlabeled thermal data (the target). The challenge is the massive "domain shift"—thermal images look nothing like RGB images. Existing UDA methods often get stuck in local optima or overfit, failing to capture the nuances of thermal signatures.
Proposed Method: Meta-UDA
The authors propose Meta-UDA, an algorithm-agnostic framework that "wraps" around existing UDA methods to make them more effective. Instead of focusing solely on the alignment process, Meta-UDA focuses on the starting point of the detector.
- The Base Architecture: The system uses a Faster R-CNN detector with a VGG16 backbone. To handle the domain shift, it employs adversarial alignment at two levels: the image level (global style/lighting) and the instance level (specific object appearance/size).
- Meta-Learning the Initial Condition: The core innovation is treating the detector's initialization as a meta-learning problem. The system learns a "starting state" specifically optimized to be highly adaptable to the thermal domain, preventing it from getting stuck in "RGB-centric" configurations.
- Online Meta-Adaptation: Standard meta-learning is computationally heavy for complex tasks like object detection. To make this practical, the authors developed an online paradigm that performs intermittent meta-updates at regular intervals during training, keeping the computation graph short and tractable.
Experimental Results
The framework was evaluated on two primary benchmarks: the KAIST Multi-Spectral Pedestrian Dataset (autonomous driving) and DSIAC (military target recognition).
- KAIST Pedestrian Detection: Meta-UDA achieved 24.6 mAP (mean Average Precision), more than doubling the 11.9 mAP managed by standard domain-adaptive Faster R-CNN.
- DSIAC (ATR): In the most challenging "Range-All" setting, Meta-UDA improved the mAP from 6.2 (standard DA) to 9.1. Qualitative results show the model successfully identifying tiny targets at 5,000 meters that were invisible to previous models.
Qualitative visualizations demonstrate that Meta-UDA significantly reduces false positives compared to source-only models and standard UDA, effectively capturing thermal signatures in low-visibility environments.
Conclusion
Meta-UDA demonstrates that meta-learning the initial state of a detector can significantly boost its ability to adapt to new domains like thermal imagery. While the "online" approach makes meta-learning tractable, the process remains more computationally demanding than standard training. Future work could explore applying this framework to modern "one-stage" detectors like YOLO or transformer-based architectures (DETR) to offer better speed and accuracy trade-offs for real-time applications like autonomous drones.
Summary updated