TV2++: A Novel Spatial-Temporal Total Variation for Super Resolution with Exponential-type Norm

Lizhen Deng, Zhetao Zhou, Guoxia Xu, Hu Zhu, Bing‐Kun Bao

Abstract

Recently, many super-resolution algorithms have been proposed to recover high resolution images to improve visualization and help better analyze images. Among them, total variation regularization (TV) methods have been proven to have a good effect in retaining image edge information. However, these TV methods do not consider the temporal correlation between images. Our algorithm designs a new TV regularization (TV2++) to take advantage of the time dimension information of the images, further improving the utilization of useful information in the images. In addition, the union of global low rank regularization and TV regularization further enhances the image super resolution recovery. And we extend the exponential-type penalty (ETP) function on singular values of a matrix to enhance low-rank matrix recovery. A novel image super-resolution algorithm based on the ETP norm and TV2++ regularization is proposed. And the alternating direction method of multipliers (ADMM) is applied to solve the optimization problems effectively. Numerous experimental results prove that the proposed algorithm is superior to other algorithms.

Introduction

Super-resolution (SR) is the process of recovering high-frequency details from low-resolution images, effectively enhancing visual quality beyond hardware constraints. While deep learning approaches are prevalent, they often require massive datasets and lack generalizability. Traditional reconstruction-based methods frequently employ Total Variation (TV) regularization to preserve sharp edges. However, standard TV methods face two significant challenges: the failure to exploit temporal correlations in image sequences (the "Temporal Gap") and the tendency to produce unnatural "staircase" artifacts in smooth gradients.

This paper introduces TV2++, a novel regularization framework that addresses these issues by decoupling spatial and temporal dimensions. By treating time and space separately, the model respects the unique characteristics of temporal data while suppressing noise more effectively.

Methodology

The proposed framework, LRTV2++, integrates three core technical components:

  1. TV2++ Regularization: Unlike standard 3D TV which couples all dimensions, TV2++ decouples spatial dimensions ($x, y$) from the temporal dimension ($t$). This flexibility allows for distinct penalization of changes across time versus space, leading to superior detail recovery.
  2. Exponential-type Penalty (ETP) Norm: To ensure global consistency, the authors utilize low-rank matrix recovery. Instead of the standard nuclear norm, they employ the non-convex ETP function as a tighter surrogate for the $L_0$ norm, more accurately recovering the underlying rank of the image data.
  3. ADMM Optimization: The resulting optimization problem is solved using the Alternating Direction Method of Multipliers (ADMM), which iteratively updates the image, low-rank components, and auxiliary variables.

The study utilized the HNSCC-3DCT-RT dataset (Head and Neck Squamous Cell Carcinoma 3D Computed Tomography) for evaluation, with low-resolution images generated via MATLAB simulations to provide ground truth.

Results and Discussion

The LRTV2++ method was compared against several benchmarks, including L1TV, L2TV, Non-Local Means (NLM), and Low-Rank Total Variation (LRTV).

Quantitative analysis demonstrates the superiority of the proposed method. In tests on the RTHEADNECK dataset, LRTV2++ achieved a Peak Signal-to-Noise Ratio (PSNR) of 32.46 dB, significantly outperforming the next best method, LRTV (29.47 dB).

Table 1: Quantitative comparison of PSNR and SSIM for various SR methods on the first test case.

Furthermore, the Structural Similarity Index (SSIM) for LRTV2++ reached 0.9846, indicating high fidelity to the original high-resolution ground truth. Similar performance advantages were observed across different test scenarios, as shown in Table 2.

Table 2: Quantitative comparison of PSNR and SSIM for various SR methods on a second test case.

The combination of global low-rank constraints and local TV2++ regularization makes the model exceptionally robust to Gaussian noise. Qualitatively, LRTV2++ produces clearer edges and fewer artifacts in CT scan enlargements, which is critical for medical diagnostics.

Conclusion

The TV2++ framework offers a significant advancement in image sequence super-resolution by effectively decoupling spatial and temporal information. While the method introduces higher computational complexity and requires parameter tuning, its superior performance in edge preservation and noise immunity makes it a powerful tool for medical imaging. Future work will focus on optimizing the ADMM implementation for speed and exploring applications in satellite imagery and high-speed video.

Summary updated

Abstract

Abstract Recently, many super-resolution algorithms have been proposed to recover high resolution images to improve visualization and help better analyze images. Among them, total variation regularization (TV) methods have been proven to have a good effect in retaining image edge information. However, these TV methods do not consider the temporal correlation between images. Our algorithm designs a new TV regularization (TV2++) to take advantage of the time dimension information of the images, further improving the utilization of useful information in the images. In addition, the union of global low rank regularization and TV regularization further enhances the image super resolution recovery. And we extend the exponential-type penalty (ETP) function on singular values of a matrix to enhance low-rank matrix recovery. A novel image super-resolution algorithm based on the ETP norm and TV2++ regularization is proposed. And the alternating direction method of multipliers (ADMM) is applied to solve the optimization problems effectively. Numerous experimental results prove that the proposed algorithm is superior to other algorithms.
BibTeX
@inproceedings{deng2020tv,
  author = {Lizhen Deng and Zhetao Zhou and Guoxia Xu and Hu Zhu and Bing‐Kun Bao},
  title = {{TV2++: A Novel Spatial-Temporal Total Variation for Super Resolution with Exponential-type Norm}},
  booktitle = {Research},
  year = {2020},
  doi = {10.21203/rs.2.23965/v2},
}
Figure