Audio-driven human body motion analysis and synthesis

Ferda Ofli, Cristian Canton-Ferrer, Joëlle Tilmanne, Yasemin Demir, Elif Bozkurt, Yucel Yemez, Engin Erzin, A. Murat TekalpICASSP 2008 · OpenReview

Abstract

This paper presents a framework for audio-driven human body motion analysis and synthesis. We address the problem in the context of a dance performance, where gestures and movements of the dancer are mainly driven by a musical piece and characterized by the repetition of a set of dance figures. The system is trained in a supervised manner using the multiview video recordings of the dancer. The human body posture is extracted from multiview video information without any human intervention using a novel marker-based algorithm based on annealing particle filtering. Audio is analyzed to extract beat and tempo information. The joint analysis of audio and motion features provides a correlation model that is then used to animate a dancing avatar when driven with any musical piece of the same genre. Results are provided showing the effectiveness of the proposed algorithm.
BibTeX
@inproceedings{ofli2008audio,
  author = {Ferda Ofli and Cristian Canton-Ferrer and Joëlle Tilmanne and Yasemin Demir and Elif Bozkurt and Yucel Yemez and Engin Erzin and A. Murat Tekalp},
  title = {{Audio-driven human body motion analysis and synthesis}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2008},
  url = {https://openreview.net/forum?id=ltMBWew84cG},
}

Loading related content...

Figure