Abstract
Thermal infrared (IR) object detection is critical for autonomous systems operating in low-light or adverse weather conditions. However, existing physical adversarial attacks on IR detectors are often limited by static materials or unidirectional heating elements. This paper introduces AdvDisplay, a reusable, bidirectional thermal display composed of a Thermoelectric Cooler (TEC) array. By leveraging PID control and a specialized optimization framework that accounts for physical heat transfer, AdvDisplay generates precise thermal patterns capable of deceiving state-of-the-art detectors like YOLOv5. Experimental results demonstrate a 96.1% attack success rate in real-world scenarios, significantly outperforming previous methods.
Introduction
Adversarial attacks in the visible spectrum, such as patches that hide pedestrians from cameras, are well-documented. In contrast, attacking thermal infrared (IR) detectors presents unique challenges. Thermal imaging captures emitted heat radiation rather than reflected light, precluding the use of standard printed patches. Current physical IR attacks are typically passive (using materials like aerogels) or active (using bulbs/resistors). Passive methods are bulky and non-reusable, while active methods often lack cooling capabilities and precise temperature control, leading to a "reality gap" where digital adversarial patterns fail when deployed physically.
Methodology: The AdvDisplay System
AdvDisplay bridges the digital-to-physical gap through a coordinated hardware-software approach.
Hardware Design
The core hardware consists of an array of Thermoelectric Coolers (TECs). Unlike traditional heaters, TECs utilize the Peltier effect to provide bidirectional temperature control—heating or cooling based on current direction. Each TEC "pixel" is managed by an STM32 microprocessor using a Proportional-Integral-Derivative (PID) control scheme to ensure rapid and stable target temperature attainment.
Optimization Framework
To generate effective adversarial patterns without internal model access, the researchers employed the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). The optimization process incorporates two critical loss functions to ensure physical realizability:
- Heat Transfer Loss ($L_{ht}$): Penalizes sharp temperature gradients between adjacent pixels to account for natural thermal diffusion.
- Electric Power Loss ($L_{ep}$): Minimizes power consumption to enhance the device's stealth and operational longevity.
Experimental Results
The system was evaluated using the FLIR ADAS V2 dataset and a fine-tuned YOLOv5 detector.
Digital and Physical Performance
In digital simulations, AdvDisplay reduced the detector's Average Precision (AP) from 100% to 41.2%. More importantly, in physical trials, the device achieved a 96.1% Attack Success Rate (ASR) at a distance of 2 meters.
Robustness and Comparisons
AdvDisplay maintained high effectiveness across various distances (1m to 5m) and viewing angles (up to $\pm 30^\circ$). It significantly outperformed existing benchmarks, such as the "Bulb Attack" (which only reduced AP to 84.4%) and "QR Attack" (64.1% AP), highlighting the importance of bidirectional thermal control and fine-grained patterns.
Conclusion
AdvDisplay represents a significant advancement in physical adversarial attacks against thermal detectors. By treating the adversarial patch as a reusable, controllable display and accounting for physical heat transfer during optimization, the researchers have exposed a critical vulnerability in IR-based sensing systems. Future work will focus on miniaturizing the hardware and exploring defensive strategies like adversarial training to mitigate these threats.
Summary updated