AnyTSR: Any-Scale Thermal Super-Resolution for UAV

Mengyuan Li, Chang–Hong Fu, Ziyu Lu, Zijie Zhang, Haobo Zuo, Liangliang Yao

TL;DR

This paper introduces AnyTSR, a novel single-model approach for any-scale thermal super-resolution specifically designed for UAVs. It tackles the limitations of fixed-scale SR methods by incorporating scale-specific information into its encoder and using an innovative upsampler that leverages coordinate offsets for improved detail reconstruction, outperforming existing methods on a new UAV thermal dataset.

What Problem Does This Paper Address?

Intelligent UAVs increasingly rely on thermal imaging for navigation and object detection in challenging environments. However, low-resolution thermal sensors produce images with insufficient detail and blurred boundaries, hindering accurate analysis. While Super-Resolution (SR) techniques can enhance image quality, most existing methods are designed for fixed scaling factors (e.g., 2x, 4x). This necessitates training separate models for each desired resolution, which is computationally expensive and inflexible for UAV applications where altitude and field-of-view changes demand adaptive scaling. Furthermore, current "any-scale" SR methods often use a uniform feature representation across all scales, failing to capture scale-specific nuances crucial for high-quality reconstruction.

Methods

The proposed AnyTSR framework consists of two main components:

  • Scale-Specific LR Image Encoder: This encoder is designed to explicitly assign unique feature codes based on the desired SR scale. It integrates scale-specific information into a state-space layer (S-SSL) architecture, which builds upon the efficiency of Mamba [25] for capturing long-term dependencies. A key innovation here is the "Scale Adaptive Mapping" (SAM) branch within the S-SSL, which dynamically learns scale-aware kernel weights. This allows the model to adapt its feature extraction process to the specific scaling requirements. The encoder also incorporates first and second-order gradients of the input image to preserve boundary and texture details.
  • Any-Scale Upsampler: This module addresses the limitations of traditional upsampling methods that rely on simple interpolation. It uses a novel approach that leverages feature offsets and scale prior information. The process involves:
  • Spatial Dimension Expansion: The latent code and coordinates are expanded to match the HR query coordinates.
  • Coordinate Offset Refinement: Radial Basis Functions (RBFs), specifically Gaussian functions, are used to compute dynamic weighting coefficients based on the distance between the query point and neighboring features. This allows for more flexible and precise weight assignment than traditional methods.
  • Image Reconstruction: A neural operator then reconstructs the HR image by integrating the weighted features, coordinate offsets, and the SR scale.

Datasets

  • UAV-TSR: A novel dataset introduced by the authors, specifically designed for UAV thermal SR tasks. It comprises 1100 high-resolution (HR) thermal images captured in land scenes, divided into 800 for training and 300 for testing.
  • Water Surface Scenes: An additional set of 200 thermal images captured over water surfaces was collected for further evaluation of the model's generalization capabilities.

Key Insights and Ideas

  • Scale-Aware Feature Encoding is Crucial: The paper strongly emphasizes that a uniform feature representation is insufficient for effective any-scale SR. By explicitly incorporating scale information into the encoder, AnyTSR can generate more accurate and flexible representations tailored to specific scaling needs.
  • Coordinate Offsets Enhance Detail Reconstruction: Traditional methods often struggle with accurately modeling the relationship between spatial coordinates and feature values. AnyTSR's upsampler effectively uses coordinate offsets, modulated by RBFs, to dynamically weight local features, leading to improved reconstruction of fine-grained details and reduced artifacts.
  • State-Space Models for Efficient SR: The adoption of state-space models (like Mamba) offers an efficient way to handle long-range dependencies in thermal images, which is beneficial for SR tasks, especially in complex scenes.
  • Single-Step Synthesis is Superior: The study demonstrates that synthesizing SR images in a single step, rather than multiple cascaded steps, leads to better performance due to the avoidance of error accumulation.

Key Results

  • State-of-the-Art Performance: AnyTSR consistently outperforms existing state-of-the-art methods (LIIF, LTE, SRNO) across various scaling factors on both land and water thermal image datasets.
  • Significant PSNR Gains: On Test set 1, at a scaling factor of x1.45, AnyTSR achieved a PSNR of 45.10 dB, surpassing Bicubic by 7.61 dB and SRNO by 0.30 dB. Even at extreme scales (x6), it achieved 30.12 dB, outperforming SRNO by 0.09 dB.
  • Robustness to Out-of-Distribution Scales: The model demonstrates strong performance even on scaling factors not explicitly included in the training range, highlighting its generalization capabilities.
  • Qualitative Improvements: Visual results show that AnyTSR generates sharper outlines, better alignment with ground truth, and preserves intricate textures more effectively than competing methods, especially in challenging regions with fine structures.

Limitations and Future Work

  • Real-time Performance: While AnyTSR offers significant improvements, further optimization for real-time performance on edge devices is a potential area for future work.
  • Lightweight Variants: The proposed framework can serve as a teacher model for knowledge distillation, enabling the development of more lightweight variants suitable for deployment on resource-constrained platforms.
  • Multimodal Fusion: The authors suggest exploring multimodal fusion applications as a future research direction, which could further enhance the capabilities of thermal SR in UAV scenarios.

Summary updated

Abstract

Thermal imaging can greatly enhance the application of intelligent unmanned aerial vehicles (UAV) in challenging environments. However, the inherent low resolution of thermal sensors leads to insufficient details and blurred boundaries. Super-resolution (SR) offers a promising solution to address this issue, while most existing SR methods are designed for fixed-scale SR. They are computationally expensive and inflexible in practical applications. To address above issues, this work proposes a novel any-scale thermal SR method (AnyTSR) for UAV within a single model. Specifically, a new image encoder is proposed to explicitly assign specific feature code to enable more accurate and flexible representation. Additionally, by effectively embedding coordinate offset information into the local feature ensemble, an innovative any-scale upsampler is proposed to better understand spatial relationships and reduce artifacts. Moreover, a novel dataset (UAV-TSR), covering both land and water scenes, is constructed for thermal SR tasks. Experimental results demonstrate that the proposed method consistently outperforms state-of-the-art methods across all scaling factors as well as generates more accurate and detailed high-resolution images. The code is located at https://github.com/vision4robotics/AnyTSR.
BibTeX
@misc{li2025anytsr,
  author = {Mengyuan Li and Chang–Hong Fu and Ziyu Lu and Zijie Zhang and Haobo Zuo and Liangliang Yao},
  title = {{AnyTSR: Any-Scale Thermal Super-Resolution for UAV}},
  year = {2025},
  eprint = {2504.13682},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.13682},
}
Figure