Composition of Motion from Video Animation Through Learning Local Transformations

Michalis Vrigkas, Virginia Tagka, Marina E. Plissiti, Christophoros NikouICASSP 2023 · OpenReview

Abstract

In this work, we solve the problem of motion representation in videos, according to local transformations applied to specific keypoints extracted from static the images. First, we compute the co-ordinates of the keypoints of the body or face through a pre-trained model, and then we introduce a convolutional neural network to estimate a dense motion field through optical flow. Next, we train a generative adversarial network that exploits the previous information to generate new images that resemble as much as possible the target frames. To reduce trembling and extract smooth movements, our model incorporates a low-pass spatio-temporal Gaussian filter. Results indicate that our method provides high performance and the movement of objects is accurate and robust.
BibTeX
@inproceedings{vrigkas2023composition,
  author = {Michalis Vrigkas and Virginia Tagka and Marina E. Plissiti and Christophoros Nikou},
  title = {{Composition of Motion from Video Animation Through Learning Local Transformations}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  url = {https://openreview.net/forum?id=G6M4exthOL},
}

Loading related content...

Figure