Image Super-Resolution Using a Simple Transformer Without Pretraining

Huan Liu, Mingwen Shao, Chao Wang, Feilong Cao

Abstract

Vision Transformer (ViT) has attracted tremendous attention and achieved remarkable success on high-level visual tasks. However, ViT relies on costly pre-training on large external datasets and is strict in data and calculations, making it an obstacle to running on common equipment. To address this challenge, we propose a simple and efficient Transformer namely SRT tailored for the image super-resolution (SR) reconstruction task. It is trained on a single GPU card without large-scale pre-training. At the beginning of the whole model, we introduce a convolutional stem module instead of straightforward tokenization of raw input images for low-level feature extraction and steady training. In the main Transformer learning phase, we exploit an additional head-convolution to make up for the lack of information interaction in multi-head self-attention (MHSA). Then further to strengthen the spatial correlation of neighboring tokens in MLP, a locally-enhanced feed-forward layer is thus employed to promote local dependencies. In terms of the inefficiency of Transformer, a channel reduction strategy is presented in MHSA, which dramatically reduces the complexity and space resources. Experimental results demonstrate the proposed Transformer model can rival the current state-of-the-art methods with a single GPU card.
BibTeX
@inproceedings{liu2023image,
  author = {Huan Liu and Mingwen Shao and Chao Wang and Feilong Cao},
  title = {{Image Super-Resolution Using a Simple Transformer Without Pretraining}},
  booktitle = {Neural Process. Lett},
  year = {2023},
  url = {https://openreview.net/forum?id=t1SiaReEG-U},
}

Loading related content...

Figure