Multimodal Continual Learning Using Online Dictionary Updating

Fuchun Sun, Huaping Liu, Chao Yang, Bin Fang

Abstract

This article studies the continual learning task from the perspective of multimodal fusion. In the multimodal fusion problem, the unified representations of heterogeneous modalities are continuously updated, while the classifier is also constantly refreshed for each multimodal learning task. To tackle this point, we establish a continual learning framework for multimodal learning and design an effective online dictionary updating method. Finally, we experimentally verify a complex material identification task and obtain promising results.
BibTeX
@article{sun2021multimodal,
  author = {Fuchun Sun and Huaping Liu and Chao Yang and Bin Fang},
  title = {{Multimodal Continual Learning Using Online Dictionary Updating}},
  journal = {IEEE Trans. Cogn. Dev. Syst.},
  year = {2021},
  url = {https://openreview.net/forum?id=NyDGIJZmp5p},
}

Loading related content...

Figure