Evolving Memory Cell Structures for Sequence Learning

Justin Bayer, Daan Wierstra, Julian Togelius, Jürgen Schmidhuber

Abstract

Long Short-Term Memory (LSTM) is one of the best recent supervised sequence learning methods. Using gradient descent, it trains memory cells represented as differentiable computational graph structures. Interestingly, LSTM’s cell structure seems somewhat arbitrary. In this paper we optimize its computational structure using a multi-objective evolutionary algorithm. The fitness function reflects the structure’s usefulness for learning various formal languages. The evolved cells help to understand crucial features that aid sequence learning.
BibTeX
@inproceedings{bayer2009evolving,
  author = {Justin Bayer and Daan Wierstra and Julian Togelius and Jürgen Schmidhuber},
  title = {{Evolving Memory Cell Structures for Sequence Learning}},
  booktitle = {ICANN},
  year = {2009},
  doi = {10.1007/978-3-642-04277-5\_76},
}

Loading related content...

Figure