HOTCOLD Block: Fooling Thermal Infrared Detectors with a Novel Wearable Design

Hui Wei, Zhixiang Wang, Xuemei Jia, Yinqiang Zheng, Hao Tang, Shin'ichi Satoh, Zheng Wang

Abstract

Thermal infrared (IR) detectors are increasingly deployed in security and autonomous systems due to their ability to operate in low-light conditions. However, these systems are vulnerable to physical adversarial attacks. This paper presents the HOTCOLD Block, a low-cost, wearable adversarial design that utilizes common warming and cooling patches to disrupt thermal heat signatures. By optimizing the size, shape, and position (SSP) of these patches using Particle Swarm Optimization (PSO), the attack achieves a significant reduction in detection accuracy while remaining stealthy under standard clothing.

Introduction

Adversarial attacks—small, intentional perturbations designed to trick machine learning models—have been extensively studied in the visible light (RGB) spectrum. In contrast, physical attacks against thermal infrared (IR) detectors face unique challenges. Unlike RGB cameras that capture reflected light, thermal cameras detect emitted heat. Consequently, an effective adversarial patch must physically alter the target's heat profile.

Previous attempts to fool thermal detectors relied on bulky equipment like incandescent bulbs or expensive, specialized materials like aerogels. These methods often suffer from a lack of stealth, appearing conspicuous to human observers, and involve complex manufacturing processes. The HOTCOLD Block addresses these limitations by providing a practical, inexpensive (<$1), and stealthy alternative.

Methodology

The HOTCOLD Block leverages the temperature difference between the human body and readily available warming and cooling pastes. The design process focuses on three primary attributes, collectively referred to as SSP:

  1. Size: The total surface area covered by the patches.
  2. Shape: Modeled via a nine-square grid to allow for various block configurations.
  3. Position: The specific placement of patches on the human torso or limbs.
Comparison of the proposed HOTCOLD block with existing bulb-based and QR-based thermal adversarial attacks.

To optimize these discrete parameters, the researchers employed Particle Swarm Optimization (PSO). PSO is a bio-inspired algorithm that simulates the movement of particles through a search space to find the configuration that most effectively minimizes the confidence score of a target detector, such as YOLOv5.

A key feature of this approach is cross-modal stealth. Because the patches are thin, they can be worn underneath standard clothing. While the clothing conceals the patches from human eyes and RGB cameras, the thermal signature "bleeds through" to the IR sensor, maintaining the adversarial effect without alerting bystanders.

Experimental Evaluation

The system was evaluated using both digital simulations and physical real-world tests.

Digital Performance

Using the Teledyne FLIR ADAS Thermal Dataset v2.0, the authors tested the attack against a YOLOv5 detector. The HOTCOLD Block reduced the Average Precision (AP) from a baseline of 94.8% to 43.0%. This performance significantly exceeded that of random patch placement or manual heuristic designs.

Physical Performance

Real-world testing was conducted using a FLIR ONE Pro camera. The attack demonstrated high robustness, achieving an Attack Success Rate (ASR) of over 90% at distances of 1 to 2 meters.

Thermal imagery demonstrating the failure of a YOLOv5 detector to identify a person wearing the HOTCOLD blocks (right) compared to a control subject (left).

Transferability and Robustness

The attack proved effective across multiple architectures, including YOLOv3, RetinaNet, and Faster R-CNN. However, the transformer-based DETR model exhibited higher resistance, suggesting that different architectural priors may offer varying levels of inherent defense against such thermal perturbations.

Discussion and Conclusion

The HOTCOLD Block demonstrates that effective physical security for thermal systems is more vulnerable than previously assumed. By moving away from complex textures and expensive materials toward simple "pure color blocks" of heat and cold, the researchers have created a highly practical threat model.

While the current design is sensitive to viewing angles and extreme distances (where resolution drops cause patches to blur), it serves as a powerful proof of concept. Future work involving 3D human modeling could further enhance the attack's multi-angle robustness. Ultimately, these findings underscore the need for adversarial training and more robust detection algorithms in the thermal infrared domain.

Summary updated

BibTeX
@inproceedings{wei2023hotcold,
  author = {Hui Wei and Zhixiang Wang and Xuemei Jia and Yinqiang Zheng and Hao Tang and Shin'ichi Satoh and Zheng Wang},
  title = {{HOTCOLD Block: Fooling Thermal Infrared Detectors with a Novel Wearable Design}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i12.26777},
}
Figure