Enabling the Encoder-Empowered GAN-based Video Generators for Long Video Generation
Lists
Only you see this
Abstract
Despite the remarkable progress in the video generation field, generating videos of longer-term remains challenging due to the challenge of sustaining the temporal consistency and continuity in the resulting synthesized movement while ensuring realism. In this paper, we propose a recall mechanism for enabling an encoder-empowered short-term video generator to produce long-term videos. This mechanism connects smoothly short video clips by modeling their temporal connections. We propose the Recall Encoder-GAN3 (REncGAN3), which enables an Encoder-based Generative Adversarial Network (GAN) to connect short generated video clips into longer sequences of hundreds of frames. The recall mechanism, defined through a loss function, enables an appropriate plasticity-continuity balance in the resulting long video stream. The proposed long-term video generation method ensures the generation of several hundred frames displaying consistent movement, which is non-repetitive while the computational memory costs are similar to those of short video generation models.
BibTeX
@inproceedings{yang2023enabling,
author = {Jingbo Yang and Adrian G. Bors},
title = {{Enabling the Encoder-Empowered GAN-based Video Generators for Long Video Generation}},
booktitle = {ICIP},
year = {2023},
doi = {10.1109/icip49359.2023.10222725},
}Loading related content...