Sample Efficiency Matters: Training Multimodal Conversational Recommendation Systems in a Small Data Setting

Haoyang Su, Wenzhe Du, Nguyen Cam-Tu, Wang XiaoliangACM Multimedia 2024 · OpenReview

Abstract

With the increasing prevalence of virtual assistants, multimodal conversational recommendation systems (multimodal CRS) becomes essential for boosting customer engagement, improving conversion rates, and enhancing user satisfaction. Yet conversational samples, as training data for such a system, are difficult to obtain in large quantities, particularly in new platforms. Motivated by this challenge, we aim to design innovative methods for training multimodal CRS effectively even in a small data setting. Specifically, assuming the availability of a small number of samples with dialogue states, we devise an effective dialogue state encoder to bridge the semantic gap between conversation and product representations for recommendation. To reduce the cost of dialogue state annotation, a semi-supervised learning method is developed to effectively train the dialogue state encoder with a small set of labeled conversations. In addition, we design a correlation regularisation that leverages knowledge in the multimodal product database to better align textual and visual modalities. Experiments on the dataset MMD demonstrate the effectiveness of our method. Particularly, with only 5% of the MMD training set, our method (namely SeMANTIC) obtains better NDCG scores than those of baseline models trained on the full MMD training set.
BibTeX
@inproceedings{su2024sample,
  author = {Haoyang Su and Wenzhe Du and Nguyen Cam-Tu and Wang Xiaoliang},
  title = {{Sample Efficiency Matters: Training Multimodal Conversational Recommendation Systems in a Small Data Setting}},
  booktitle = {ACM Multimedia},
  year = {2024},
  url = {https://openreview.net/forum?id=x79oczoigv},
}

Loading related content...

Figure