What Limits Thermal Infrared Drone Detection? A Cross-Dataset Benchmark Analysis

Tamara R. Lenhard, Jan Kels, Tobias KocharXiv 2026 · arXiv

TL;DR

Thermal infrared (IR) drone detection is currently bottlenecked more by inconsistent dataset quality and undocumented sensor artifacts than by model architecture. This benchmark reveals that even state-of-the-art YOLO models fail to detect over 40% of targets, with performance plummeting during "thermal crossover" events and in cluttered urban environments.

What Problem Does This Paper Address?

While visible-spectrum (RGB) cameras are the standard for drone surveillance, they are rendered useless by darkness or fog. Thermal infrared (IR) imaging offers a robust alternative for 24/7 monitoring. However, the field of IR drone detection is currently a "wild west" of inconsistent data.

Public datasets vary wildly in terms of camera sensors (cooled vs. uncooled), image processing (tone mapping, sharpening), and annotation practices. This makes it nearly impossible to tell if a new model is actually better or if it has simply learned to exploit specific artifacts—like on-screen display (OSD) text or sensor-specific noise—unique to a single dataset. The authors aim to systematically characterize these variabilities and establish a rigorous benchmark for cross-dataset generalization.

Methods

The researchers approached the problem through three main pillars:

  1. Artifact Taxonomy: They established a taxonomy of visual artifacts based on their origin in the imaging pipeline: Optics & Sensor (e.g., motion blur, vignetting), Tone Mapping (e.g., histogram equalization), and Recording (e.g., OSD overlays, compression).
  2. Model Benchmarking: They evaluated two generations of the popular "You Only Look Once" architecture—YOLOv8 and YOLO11—across various scales (nano to large) to see if architectural improvements could overcome data limitations.
  3. Cross-Dataset Evaluation: Instead of just testing on a "held-out" portion of the same dataset, they performed cross-domain testing: training on one dataset (e.g., CST Anti-UAV) and testing on entirely different ones (e.g., LRDDv3) to measure real-world robustness.
Figure 1. Selected examples of artifacts along the imaging pipeline in public IR drone detection and tracking datasets (left to right): Anti- UAV [10], USC [4], LRDDv3 [19], InfraTec [3], and CST [31]. Top: full-frame images with crop regions outlined in red (distinct from GT bounding boxes). Bottom: corresponding detail crops. Colors indicate stage of the imaging pipeline at which each artifact originates.

Datasets

The authors reviewed nine publicly available datasets (summarized in Figure 4/Table 1) and focused their deep-dive analysis on three major Long-Wave Infrared (LWIR) benchmarks:

  • Hamburg2 (H2): Features scene-matched subsets from two different cameras (FLIR and InfraTec).
  • LRDDv3: An air-to-air dataset with a high proportion of very small targets.
  • CST Anti-UAV: A large-scale dataset (238k frames) with diverse backgrounds and detailed attribute labels like thermal crossover.

The datasets show significant differences in target size distributions, with LRDDv3 and CST being dominated by "extra-extra-small" (XXS) targets.

Figure 3. Normalized object-size distributions for H2 [3], LRDD- v3 [19], and CST [31], showing annotation proportions by object- size category (Tab. I, Supp.) at 640×640 YOLO input resolution.

Key Insights and Ideas

  • Data Composition Trumps Model Capacity: One of the most striking findings is that detection performance is governed much more strongly by the training data than by the specific YOLO generation or model size. Moving from a "small" to a "large" model yields marginal gains compared to the massive performance swings caused by switching training sets.
  • Asymmetric Generalization: Models do not transfer well between datasets. For example, a model trained on the grayscale CST dataset fails almost completely on the pseudocolor-mapped LRDDv3 dataset. This highlights how "baked-in" image processing choices can break a detector in the field.
  • The "Thermal Crossover" Trap: Performance drops significantly during thermal crossover (TC)—periods where the drone's temperature matches the background, leading to near-zero contrast. This is a fundamental physical limit that current deep learning models struggle to navigate.
  • Shortcut Learning: The authors found evidence that models might be learning "shortcuts." For instance, OSD overlays (text on screen) can act as spurious cues that confuse the model or lead to false positives if the model associates text-heavy regions with targets.
Figure 2. Proportion of annotated drone instances exhibiting TC, DBC, and CDB across all splits of CST [31].

Key Results

  • High Miss Rates: Even the best-performing configurations had a False Negative Rate (FNR) exceeding 40%. In critical infrastructure protection, missing 4 out of 10 drones is often unacceptable.
  • Small Target Failure: For XXS targets (area < 8² pixels), the miss rate is particularly high, as these targets often fall below the stride of the model's finest feature maps.
  • mAP Scores: In-domain [email protected] scores reached up to 0.747 (on CST), but cross-domain scores frequently plummeted below 0.10, illustrating a severe lack of robustness.
  • YOLO11 vs. YOLOv8: YOLO11 showed slight improvements in efficiency, achieving comparable mAP to YOLOv8l with significantly fewer parameters and FLOPs, but it did not solve the underlying generalization issues.

Limitations and Future Work

The authors acknowledge that existing datasets are often poorly documented regarding their imaging pipelines (e.g., whether Automatic Gain Control was active). This lack of metadata prevents researchers from properly accounting for domain shifts.

Future Directions: * Synthetic Data: The authors suggest using synthetic data generation to systematically fill gaps in existing datasets, such as specific weather conditions or rare drone models. * Radiometric Data: Moving toward raw radiometric data (temperature values) rather than 8-bit "visualized" IR images could eliminate the biases introduced by tone mapping and pseudocolor schemes. * Standardized Annotations: There is a dire need for consistent bounding-box conventions (e.g., whether to include motion-blurred propellers) to ensure fair evaluation across the community.

Summary updated

Abstract

Thermal infrared (IR) imaging enables drone detection with reduced dependence on ambient illumination, supporting continuous surveillance beyond the limitations of visible-spectrum sensing. However, substantial and often undocumented variability in camera systems, processing pipelines, and annotation practices across public IR drone datasets complicates the assessment of detector performance and cross-domain generalization. To examine these sources of variability and their implications for drone detection, we systematically characterize existing IR datasets and establish an artifact taxonomy reflecting their origins across the acquisition and processing pipeline, encompassing optical and sensor effects, tone mapping and image enhancement, and recording-related artifacts. Focusing on long-wave IR data, we investigate the impact of training data composition and model characteristics on YOLO-based detection performance, generalization, and condition-dependent failure modes. Our results demonstrate that detection performance is governed more strongly by training data composition than by YOLO generation or model capacity, while cross-dataset generalization exhibits pronounced asymmetries. Even the best-performing configuration fails to detect over 40\% of drones, with elevated miss rates for very small targets, cluttered urban backgrounds, and thermal crossover, highlighting the shortcomings of existing datasets.
BibTeX
@misc{lenhard2026what,
  author = {Tamara R. Lenhard and Jan Kels and Tobias Koch},
  title = {{What Limits Thermal Infrared Drone Detection? A Cross-Dataset Benchmark Analysis}},
  year = {2026},
  eprint = {2611.16724},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16724},
}
Figure