A Cognitive-inspired Multi-Module Architecture for Continual Learning

Shruthi Gowda, Bahram Zonooz, Elahe AraniICLR 2023 · OpenReview

Abstract

Artificial neural networks (ANNs) exhibit a narrow scope of expertise on stationary independent data. However, data in the real world is continuous and dynamic, and ANNs must adapt to novel scenarios while also retaining the learned knowledge to become lifelong learners. The ability of humans to excel at these tasks can be attributed to multiple factors ranging from cognitive computational structures, cognitive biases, and the multi-memory systems in the brain. We incorporate key concepts from each of these to design a cognitive-inspired continual learning method. Cognitive Continual Learner (CCL) includes multiple modules, implicit and explicit knowledge representation dichotomy, inductive bias, and a multi-memory system. CCL shows improvement across all continual learning settings and also exhibits reduced task recency bias. To test versatility of continual learning methods on a challenging distribution shift, we introduce a novel domain-incremental dataset Domain${^2}$IL. In addition to improved performance on existing benchmarks, CCL also demonstrates superior performance on this dataset.
BibTeX
@inproceedings{gowda2023cognitive,
  author = {Shruthi Gowda and Bahram Zonooz and Elahe Arani},
  title = {{A Cognitive-inspired Multi-Module Architecture for Continual Learning}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=wPLEzBcSC7p},
}

Loading related content...

Figure