Learning Like a Toddler: Watching Television Series to Learn Vocabulary from Images and Audio
Lists
Only you see this
Abstract
This paper presents the initial findings of our efforts to build an unsupervised multimodal vocabulary learning scheme in a realistic scenario. For this purpose, a new multimodal dataset, called Musti3D, has been created. The Musti3D database contains episodes from an animation series for toddlers. Annotated with audiovisual information, this database is used for the investigation of a non-negative matrix factorization (NMF)-based audiovisual learning technique. The performance of the technique, i.e. correctly matching the audio and visual representations of the objects, has been evaluated by gradually reducing the level of supervision starting from the ground truth transcriptions. Moreover, we have performed experiments using different visual representations and time spans for combining the audiovisual information. The preliminary results show the feasibility of the proposed audiovisual learning framework.
BibTeX
@inproceedings{yilmaz2014learning,
author = {Emre Yilmaz and Konstantinos Rematas and Tinne Tuytelaars and Hugo Van hamme},
title = {{Learning Like a Toddler: Watching Television Series to Learn Vocabulary from Images and Audio}},
booktitle = {ACM Multimedia},
year = {2014},
doi = {10.1145/2647868.2655036},
}Loading related content...