Deep Features for CBIR with Scarce Data using Hebbian Learning

Gabriele Lagani, Davide Bacciu, Claudio Gallicchio, Fabrizio Falchi, Claudio Gennaro, Giuseppe Amato

Introduction

Content-Based Image Retrieval (CBIR) systems rely on the ability of Deep Neural Networks (DNNs) to extract meaningful feature representations from images. While DNNs have achieved state-of-the-art performance in this domain, their success is typically predicated on the availability of massive, labeled datasets. In many practical scenarios—such as categorizing images from niche events or specialized medical databases—labeled data is a scarce resource. This paper explores a semi-supervised learning strategy designed to bridge the "sample efficiency" gap, enabling effective image retrieval even when only a fraction of the dataset is labeled.

Methodology

The authors propose a two-phase learning framework that combines biologically inspired unsupervised learning with traditional supervised fine-tuning. This approach aims to leverage the structure of the entire dataset before any labels are introduced.

1. Unsupervised Hebbian Pre-training

The first phase utilizes a nonlinear Hebbian Principal Component Analysis (HPCA) learning rule. Unlike standard backpropagation, which relies on a global error signal to update weights, Hebbian learning is a local, bio-inspired mechanism often summarized by the phrase "neurons that fire together, wire together." By applying this rule, the network's internal layers learn to represent the underlying statistical structure of the data without requiring human-provided labels. This phase allows the model to develop a robust "visual vocabulary" based on the total available image pool.

2. Supervised SGD Fine-tuning

Following the unsupervised phase, the network is fine-tuned using the limited labeled data available. This is performed via standard Stochastic Gradient Descent (SGD). By starting from the Hebbian-initialized weights rather than random initialization, the network is better positioned to converge on useful features despite the small number of training examples.

To isolate the effects of these learning rules, the researchers employed a 6-layer architecture inspired by AlexNet, allowing for a granular, layer-by-layer analysis of feature extraction performance.

Experimental Setup

The methodology was evaluated on two benchmark datasets: CIFAR10 (60,000 images, 10 classes) and CIFAR100 (60,000 images, 100 classes). To simulate real-world data scarcity, the authors tested various "sample efficiency regimes," varying the amount of labeled data from 1% to 100% of the total dataset.

Key Insights and Results

The primary metric for evaluation was mean Average Precision (mAP). The results indicate that Hebbian pre-training provides a significant advantage in data-poor environments:

  • Performance in Scarce Regimes: On CIFAR10 with only 1% labels, the Hebbian approach achieved 17.81% mAP, notably outperforming both random initialization (14.17%) and Variational Auto-Encoder (VAE) pre-training (13.79%). Similar relative gains were observed on the more complex CIFAR100 dataset.
  • Comparison with VAEs: While VAEs are a common choice for unsupervised pre-training, this study found that HPCA-based pre-training is more effective when labels are extremely scarce (below 10%).
  • The Detrimental Threshold: The benefit of Hebbian pre-training is most pronounced when data is limited. Interestingly, as labeled data approaches 100%, the advantage diminishes. In high-data regimes, the specific "bias" introduced by unsupervised pre-training can occasionally hinder the network from reaching the absolute global optimum compared to pure backpropagation from scratch.
  • Optimal Layer Selection: The study found that features from Layer 3 were consistently the most robust for retrieval tasks in low-data scenarios. This suggests that middle-level features provide the ideal balance between low-level visual patterns and high-level semantic meaning.

Conclusion and Future Work

This research demonstrates that Hebbian learning is a powerful tool for extracting general knowledge from unlabeled data, providing a superior starting point for CBIR tasks when labels are scarce. While the current study focused on low-resolution (32x32) benchmarks, it establishes a foundation for more efficient semi-supervised learning.

Future research directions include scaling the approach to high-resolution datasets and exploring more complex Hebbian variants, such as Independent Component Analysis (ICA) or sparse coding. Additionally, the authors suggest that integrating these rules into Spiking Neural Networks (SNNs) could further improve biological plausibility and energy efficiency in retrieval systems.

Summary updated

BibTeX
@inproceedings{lagani2022deep,
  author = {Gabriele Lagani and Davide Bacciu and Claudio Gallicchio and Fabrizio Falchi and Claudio Gennaro and Giuseppe Amato},
  title = {{Deep Features for CBIR with Scarce Data using Hebbian Learning}},
  booktitle = {CBMI},
  year = {2022},
  doi = {10.1145/3549555.3549587},
}
Figure