Multi-modal speaker diarization of real-world meetings using compressed-domain video features

Gerald Friedland, Hayley Hung, Chuohao Yeo

Abstract

Speaker diarization is originally defined as the task of determining ldquowho spoke whenrdquo given an audio track and no other prior knowledge of any kind. The following article shows a multi-modal approach where we improve a state-of-the-art speaker diarization system by combining standard acoustic features (MFCCs) with compressed domain video features. The approach is evaluated on over 4.5 hours of the publicly available AMI meetings dataset which contains challenges such as people standing up and walking out of the room. We show a consistent improvement of about 34% relative in speaker error rate (21% DER) compared to a state-of-the-art audio-only baseline.
BibTeX
@inproceedings{friedland2009multi,
  author = {Gerald Friedland and Hayley Hung and Chuohao Yeo},
  title = {{Multi-modal speaker diarization of real-world meetings using compressed-domain video features}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2009},
  doi = {10.1109/icassp.2009.4960522},
}

Loading related content...

Figure