Word learning in a multimodal environment

Deb Roy, Alex Pentland

Abstract

We are creating human machine interfaces which let people communicate with machines using natural modalities including speech and gesture. A problem with current multimodal interfaces is that users are forced to learn the set of words and gestures which the interface understands. We report on a trainable interface which lets the user teach the system words of their choice through natural multimodal interactions.
BibTeX
@inproceedings{roy1998word,
  author = {Deb Roy and Alex Pentland},
  title = {{Word learning in a multimodal environment}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {1998},
  doi = {10.1109/icassp.1998.679702},
}

Loading related content...

Figure