Abstract
Few-shot image classification remains a significant challenge for deep learning models, which often overfit to spurious background correlations when data is scarce. This paper proposes a biologically-inspired solution by modeling the tuning characteristics of neurons in the human brain's ventral stream. By introducing Hierarchical Feature Regularization (HFR) and an unsupervised foreground segmentation technique to simulate object "belongingness," the authors significantly improve the accuracy and robustness of existing few-shot learning models. Evaluations on standard benchmarks like CUB and miniImageNet show substantial performance gains, setting new state-of-the-art results.
Introduction
Humans possess a remarkable ability to recognize new visual concepts from just a few examples. This capability is largely attributed to the ventral stream, a visual pathway that acts as a robust, generic encoder for transform-invariant features such as shape and color. In contrast, standard deep learning models trained on limited data often learn "shortcut" features—like specific background textures or lighting conditions—rather than the intrinsic properties of the objects themselves.
This research addresses the gap by asking whether mathematical models of visual neurons can serve as effective priors for neural networks. By forcing a network to mimic the hierarchical processing of the human visual cortex, the authors aim to induce more human-like, robust feature extraction.
Methodology
The proposed approach is model-agnostic and consists of two primary components: Hierarchical Feature Regularization and Foreground Segmentation.
Hierarchical Feature Regularization (HFR)
The authors mathematically model the tuning characteristics of neurons across three key areas of the ventral stream hierarchy:
- V1 Area: Modeled using simple and complex cells tuned to specific edge orientations, alongside double-opponent cells for color contrast.
- V2 Area: Modeled using endstopped cells tuned to the degree and sign of curvature.
- V4 Area: Modeled using cells tuned to complex contour fragments and hue (invariant to luminance).
These fixed, computational models are used to regularize the intermediate layers of a standard CNN backbone (e.g., ResNet-12) via Mean Squared Error. The regularization is applied hierarchically: shallow layers are aligned with V1 models, middle layers with V2, and deeper layers with V4. This ensures the network builds feature complexity in an order consistent with biological systems.
Foreground Segmentation for "Belongingness"
Biological neurons exhibit "belongingness," firing more intensely for foreground objects than for backgrounds. To simulate this, the authors developed an unsupervised segmentation algorithm. Assuming that objects rarely occupy the extreme edges of an image, the system uses K-means clustering on the HSV color features of edge patches to define the "background." A foreground mask is then generated and multiplied with the network's feature maps to suppress background noise and prevent shortcut learning.
Experimental Results
The method was evaluated on four major few-shot benchmarks: CUB (fine-grained birds), miniImageNet, tieredImageNet, and CIFAR-FS.
Performance Gains
The proposed "ventral" variants consistently outperformed baseline models across various training paradigms:
- CUB Dataset: Achieved a significant +3.84% boost on 5-way 1-shot classification using the RENet backbone, rising from 79.49% to 83.33%.
- miniImageNet: Improved 1-shot RENet accuracy by +2.11% (67.60% to 69.71%).
- State-of-the-Art: The "RENet-ventral" and "LR-DC-ventral" models achieved SOTA results on tieredImageNet (79.48% for 1-shot).
Key Insights
- Hierarchy is Essential: Ablation studies revealed that reversing the regularization hierarchy (e.g., V4 models on shallow layers) degrades performance. The network must build complexity in the same sequence as the brain.
- Shape vs. Color: Tuning for shape cues (edges and curves) provided a more substantial performance boost than color or belongingness alone.
- Regularization vs. Distillation: Direct feature regularization (HFR) proved more effective than traditional knowledge distillation from a teacher model.
Conclusion and Future Work
By bridging computational neuroscience and deep learning, this work demonstrates that biological priors can effectively mitigate overfitting in few-shot scenarios. While the current "belongingness" heuristic is effective, it remains sensitive to image resolution and framing, as seen in the more modest gains on the low-resolution CIFAR-FS dataset. Future research may explore more sophisticated, learned neural representations for edge regions to further refine foreground-background separation.
Summary updated