Recurrent Layer Aggregation using LSTM

Yu Qin, Jiajun Du, Xinyao Wang, Hongtao Lu

Abstract

Standard convolutional neural networks assemble multiple convolutional layers to extract high-level features. Recent efforts keep designing deeper and wider architectures. Even with skip connections applied to combine different layers, the useful low-level features are not effectively utilized. Some deep layer aggregation methods have been proposed to aggregate features of all levels, using simple linear combination or complex non-linear transformation. In this paper, we treat convolutional features as a sequence, and propose our Recurrent Aggregation of Convolutional Neural Network (CNN-RA). Our aggregation method splits a standard CNN into blocks and maps their feature matrices to a sequence of vectors of the same length. LSTM is employed to connect to the sequence and better fuse features across blocks. Our proposed CNN-RA can be directly appended to any standard CNN without any modifications. Experiments show remarkable improvements of CNN-RA over the original architectures across datasets.
BibTeX
@inproceedings{qin2019recurrent,
  author = {Yu Qin and Jiajun Du and Xinyao Wang and Hongtao Lu},
  title = {{Recurrent Layer Aggregation using LSTM}},
  booktitle = {IJCNN},
  year = {2019},
  doi = {10.1109/ijcnn.2019.8852077},
}

Loading related content...

Figure