Multi-Speaker Tracking From an Audio-Visual Sensing Device
Lists
Only you see this
Abstract
Compact multi-sensor platforms are portable and thus desirable for robotics and personal-assistance tasks. However, compared to physically distributed sensors, the size of these platforms makes person tracking more difficult. To address this challenge, we propose a novel 3-D audio-visual people tracker that exploits visual observations (object detections) to guide the acoustic processing by constraining the acoustic likelihood on the horizontal plane defined by the predicted height of a speaker. This solution allows the tracker to estimate, with a small microphone array, the distance of a sound. Moreover, we apply a color-based visual likelihood on the image plane to compensate for misdetections. Finally, we use a 3-D particle filter and greedy data association to combine visual observations, color-based, and acoustic likelihoods to track the position of multiple simultaneous speakers. We compare the proposed multimodal 3-D tracker against two state-of-the-art methods on the AV16.3 dataset and on a newly collected dataset with co-located sensors, which we make available to the research community. Experimental results show that our multimodal approach outperforms the other methods both in 3-D and on the image plane.
BibTeX
@article{qian2019multi,
author = {Xinyuan Qian and Alessio Brutti and Oswald Lanz and Maurizio Omologo and Andrea Cavallaro},
title = {{Multi-Speaker Tracking From an Audio-Visual Sensing Device}},
journal = {IEEE Trans. Multimedia},
year = {2019},
url = {https://openreview.net/forum?id=wlk1Pqc4V4V},
}Loading related content...