SIMILARnet: Simultaneous Intelligent Localization and Recognition Network

Arna Ghosh, Biswarup Bhattacharya, Somnath Basu Roy ChowdhuryarXiv 2017 · arXiv

Abstract

Traditional Convolutional Neural Networks (CNNs) often require architectural modifications or additional training cycles to perform object localization alongside classification. This paper introduces SIMILARnet, a biologically inspired architecture that achieves simultaneous localization and recognition without extra training. By quantifying the importance of individual convolutional filters through performance degradation analysis, the network generates accurate localization heatmaps for images of arbitrary dimensions.

Introduction

While CNNs excel at image classification, the transition from convolutional layers to fully connected layers typically results in the loss of spatial information. Existing techniques like Class Activation Mapping (CAM) utilize Global Average Pooling (GAP) to recover this information, but they often necessitate "differential connections"—re-training or modifying the network specifically for the localization task. SIMILARnet addresses this by extracting localization data directly from a pre-trained classification network, mimicking biological neural structures that remain relatively stable after development.

Methodology

The SIMILARnet framework is built upon a Le-Net foundation. The core innovation is the method used to determine the "importance" of convolutional filters for generating a localization heatmap.

SIMILARnet architecture overview showing the flow from input image through convolutional layers to the final localization mask.

The process involves five key steps:

  1. Baseline Error ($E$): Recording the classification error of the fully trained network.
  2. Filter Perturbation ($e_i$): Systematically "blocking" (zeroing out) each filter in the final convolutional layer and recording the new error.
  3. Weight Calculation: Defining importance weights ($w'_i$) as the difference between baseline and perturbed error ($w'_i = E - e_i$). A significant spike in error indicates a critical feature.
  4. Normalization: Applying a softmax operation to ensure weights sum to unity.
  5. Heatmap Generation ($H$): Creating the final localization map as a weighted linear combination of feature maps: $H = \sum w_i \times F_i$.

This approach allows the model to process images of various sizes since it relies on inherent filter properties rather than fixed-size fully connected layers.

Experimental Results

The model was evaluated on the MNIST dataset and various natural images containing text.

Classification Performance

On the MNIST test set, SIMILARnet achieved a classification accuracy of 98.73%. This high accuracy demonstrates that the localization mechanism does not compromise the network's primary recognition capabilities.

Localization in Natural Scenes

Beyond standard digit recognition, the network was tested on complex natural images. Despite being trained solely on MNIST, the model successfully localized text in diverse environments, such as street signs and house numbers.

Localization results on a natural image containing the text "TOP 10 CITIES", showing the original image (g) and the generated localization mask (h).

The results suggest that the low-level features learned for digit recognition (edges, curves) generalize well to broader character localization tasks.

Discussion and Conclusion

SIMILARnet provides a zero-overhead method for object localization. By using the network's own performance degradation as a proxy for feature importance, it avoids the need for secondary training phases. This approach is more biologically plausible than methods requiring constant re-weighting for different sub-tasks.

Future research will focus on scaling this logic to deeper architectures like ResNet and Inception, as well as refining the method to produce class-specific heatmaps for multi-object localization in complex scenes.

Summary updated

Abstract

Global Average Pooling (GAP) [4] has been used previously to generate class activation for image classification tasks. The motivation behind SIMILARnet comes from the fact that the convolutional filters possess position information of the essential features and hence, combination of the feature maps could help us locate the class instances in an image. We propose a biologically inspired model that is free of differential connections and doesn't require separate training thereby reducing computation overhead. Our novel architecture generates promising results and unlike existing methods, the model is not sensitive to the input image size, thus promising wider application. Codes for the experiment and illustrations can be found at: https://github.com/brcsomnath/Advanced-GAP .
BibTeX
@misc{ghosh2017similarnet,
  author = {Arna Ghosh and Biswarup Bhattacharya and Somnath Basu Roy Chowdhury},
  title = {{SIMILARnet: Simultaneous Intelligent Localization and Recognition Network}},
  year = {2017},
  eprint = {1711.02831},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1711.02831},
}
Figure