Segmentation and Shape Extraction from Convolutional Neural Networks

Mai Lan Ha, Gianni Franchi, Michael Möller, Andreas Kolb, Volker Blanz

Abstract

We propose a novel method for creating high-resolution class activation maps from a given deep convolutional neural network which was trained for image classification. The resulting class activation maps not only provide information about the localization of the main objects and their instances in the image, but are also accurate enough to predict their shapes. Rather than pursuing a weakly supervised learning strategy, the proposed algorithm is a multiscale extension of the classical class activation maps using a principal component analysis of the classification network feature maps, guided filtering, and a conditional random field. Nevertheless, the resulting shape information is competitive with state-of-the-art weakly supervised segmentation methods on datasets on which the latter have been trained, while being significantly better at generalizing to other datasets and unknown classes.
BibTeX
@inproceedings{ha2018segmentation,
  author = {Mai Lan Ha and Gianni Franchi and Michael Möller and Andreas Kolb and Volker Blanz},
  title = {{Segmentation and Shape Extraction from Convolutional Neural Networks}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2018},
  doi = {10.1109/wacv.2018.00169},
}

Loading related content...

Figure