DanceU: motion-and-music-based automatic effect generation for dance videos

Yanjie Pan, Yaru Du, Shandong Wang, Yun Ye, Yong Jiang, Zhen Zhou, Li Xu, Ming Lu, Yunbiao Lin, Jiehui LuICME 2023 · OpenReview

Abstract

We propose a novel method, named DanceU, to automate dance video editing in an intelligent way. Our method fully leverages two folds of a dance video, the background music and the dancer’s motion, to determine where to add effects and what kind of effects to be added. We parameterize dance motion into two metrics: motion amplitude and motion intensity, to represent the motion characteristics of different dance types. Then, we design algorithms to align the dance motions with music beats and automatically decide dance effects for the dance video. To our best knowledge it is the first proposed method for automatic dance video effect generation. Results show that our method can generate fluent, performative, and music-matched effects, which achieves similar editing level to professionals. (A demo video: https://f.io/fgQUxehZ)
BibTeX
@inproceedings{pan2023danceu,
  author = {Yanjie Pan and Yaru Du and Shandong Wang and Yun Ye and Yong Jiang and Zhen Zhou and Li Xu and Ming Lu and Yunbiao Lin and Jiehui Lu},
  title = {{DanceU: motion-and-music-based automatic effect generation for dance videos}},
  booktitle = {ICME},
  year = {2023},
  url = {https://openreview.net/forum?id=0EjXR1oUzR0},
}

Loading related content...

Figure