STNet: An End-to-End Generative Framework for Synthesizing Spatiotemporal Super-Resolution Volumes

Jun Han, Hao Zheng, Danny Z. Chen, Chaoli WangIEEE TVCG 2022 · OpenReview

Abstract

We present STNet, an end-to-end generative framework that synthesizes spatiotemporal super-resolution volumes with high fidelity for time-varying data. STNet includes two modules: a generator and a spatiotemporal discriminator. The input to the generator is two low-resolution volumes at both ends, and the output is the intermediate and the two-ending spatiotemporal super-resolution volumes. The spatiotemporal discriminator, leveraging convolutional long short-term memory, accepts a spatiotemporal super-resolution sequence as input and predicts a conditional score for each volume based on its spatial (the volume itself) and temporal (the previous volumes) information. We propose an unsupervised pre-training stage using cycle loss to improve the generalization of STNet. Once trained, STNet can generate spatiotemporal super-resolution volumes from low-resolution ones, offering scientists an option to save data storage (i.e., sparsely sampling the simulation output in both spatial and temporal dimensions). We compare STNet with the baseline bicubic+linear interpolation, two deep learning solutions ( <inline-formula><tex-math notation="LaTeX">$\mathsf{SSR}+\mathsf{TSF}$</tex-math></inline-formula> , STD), and a state-of-the-art tensor compression solution (TTHRESH) to show the effectiveness of STNet.
BibTeX
@article{han2022stnet,
  author = {Jun Han and Hao Zheng and Danny Z. Chen and Chaoli Wang},
  title = {{STNet: An End-to-End Generative Framework for Synthesizing Spatiotemporal Super-Resolution Volumes}},
  journal = {IEEE Trans. Vis. Comput. Graph.},
  year = {2022},
  url = {https://openreview.net/forum?id=BL3YMoGHIIe},
}

Loading related content...

Figure