Hybrid Function Sparse Representation towards Image Super Resolution

Junyi Bian, Baojun Lin, Ke Zhang

Abstract

Sparse representation with training-based dictionaries has been successful in super-resolution (SR) but faces limitations regarding flexibility and data dependence. We propose Hybrid Function Sparse Representation (HFSR), which utilizes a function-based dictionary generated from preset mathematical functions—approximated Heaviside (AHF), sine, and Discrete Cosine Transform (DCT)—without requiring external training. This dictionary is inherently scalable, allowing for upscaling to any size. We further introduce a multi-scale refinement strategy and a weighted reconstruction method to handle overlapping patches. Experimental results on the Set14 dataset demonstrate that HFSR achieves excellent performance, particularly on images with rich details and textures, outperforming state-of-the-art non-learning methods.

1. Introduction

Single Image Super Resolution (SISR) aims to reconstruct a high-resolution (HR) image from a single low-resolution (LR) input. This is an ill-posed problem as multiple HR images can correspond to the same LR observation. Traditional methods like bicubic interpolation are fast but often result in blurry edges. Learning-based methods, particularly deep learning, offer high quality but are constrained by the need for massive training datasets and are often fixed to specific upscaling factors.

Reconstruction-based methods using sparse representation offer a middle ground. However, they typically rely on dictionaries learned from external image patches. The proposed Hybrid Function Sparse Representation (HFSR) addresses these limitations by replacing learned dictionaries with mathematically defined, scalable function-based dictionaries.

2. Methodology

The core of HFSR is the use of a Function-based Dictionary. Instead of learning features from data, the dictionary is constructed using three types of mathematical functions:

  • Approximated Heaviside Function (AHF): Captures sharp edges and step-like transitions.
  • Sine Function: Represents periodic patterns and stripes.
  • Discrete Cosine Transform (DCT): Captures general textures and low-frequency components.
The hybrid dictionary atoms: (a) AHF, (b) DCT, and (c) Sine functions, shown at original and 2x upscaled scales.

Because these functions are continuous, the dictionary atoms can be scaled to any arbitrary factor by simply adjusting the coordinate parameters. This eliminates the need for retraining for different magnification levels.

The HFSR workflow involves three main stages:

  1. Sparse Coding: Each LR patch is represented as a sparse linear combination of dictionary atoms.
  2. Multi-scale Refinement: An iterative process that fits the residual error at multiple scales, improving reconstruction stability and fidelity.
  3. Weighted Reconstruction: To assemble the final HR image, overlapping patches are combined using weights derived from their reconstruction loss, ensuring sharper transitions and fewer artifacts.
The HFSR workflow: (a) patch extraction, (b) sparse coding using hybrid dictionaries, (c) multi-scale refinement, and (d) final HR reconstruction.

3. Experiments and Results

The authors evaluated HFSR on the standard Set14 benchmark. Comparisons were made against baseline interpolation methods (Nearest Neighbor, Bicubic) and state-of-the-art reconstruction methods like Glasner et al.

Quantitative results show that HFSR (multi-scale) consistently achieves higher Peak Signal-to-Noise Ratio (PSNR) values than other non-learning methods. For instance, on the "Zebra" image, HFSR reached 31.59 dB, significantly higher than bicubic interpolation (30.68 dB). Qualitatively, HFSR excels at recovering fine, repetitive textures such as stripes and comic book details, which are often blurred by standard methods.

4. Conclusion

HFSR presents a flexible and effective approach to SISR by leveraging the scalability of mathematical functions. By avoiding the need for external training data, it offers a versatile solution that can adapt to any upscaling factor. Future work may involve expanding the dictionary with more diverse function types or integrating learning-based components to further refine the function parameters.

Summary updated

Abstract

Sparse representation with training-based dictionary has been shown successful on super resolution(SR) but still have some limitations. Based on the idea of making the magnification of function curve without losing its fidelity, we proposed a function based dictionary on sparse representation for super resolution, called hybrid function sparse representation (HFSR). The dictionary we designed is directly generated by preset hybrid functions without additional training, which can be scaled to any size as is required due to its scalable property. We mixed approximated Heaviside function (AHF), sine function and DCT function as the dictionary. Multi-scale refinement is then proposed to utilize the scalable property of the dictionary to improve the results. In addition, a reconstruct strategy is adopted to deal with the overlaps. The experiments on Set14 SR dataset show that our method has an excellent performance particularly with regards to images containing rich details and contexts compared with non-learning based state-of-the art methods.
BibTeX
@misc{bian2019hybrid,
  author = {Junyi Bian and Baojun Lin and Ke Zhang},
  title = {{Hybrid Function Sparse Representation towards Image Super Resolution}},
  year = {2019},
  eprint = {1906.04363},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1906.04363},
}
Figure