Beyond Text and Back Again

Desmond ElliottWWW 2021 · OpenReview

Abstract

A talk with two parts covering three modalities. In the first part, I will talk about NLP Beyond Text, where we integrate visual context into a speech recognition model and find that the recovery of different types of masked speech inputs is improved by fine-grained visual grounding against detected objects [2]. In the second part, I will come Back Again, and talk about the benefits of textual supervision in cross-modal speech–vision retrieval models [1].
BibTeX
@inproceedings{elliott2021beyond,
  author = {Desmond Elliott},
  title = {{Beyond Text and Back Again}},
  booktitle = {The Web Conference},
  year = {2021},
  url = {https://openreview.net/forum?id=ovjCY5xA-A},
}

Loading related content...

Figure