Abstract
Inspired by the human brain's ability to emphasize discriminative parts of an input while suppressing irrelevant noise, researchers have developed "local mechanisms" to advance computer vision. These techniques focus on target regions to learn discriminative representations and process information selectively for improved efficiency. This survey provides a systematic review of local mechanisms across various tasks, including fine-grained visual recognition, person re-identification, few-shot learning, and Vision Transformers (ViTs). By analyzing the advantages and disadvantages of different categories, we shed light on future research directions in the field.
Introduction: The Roadmap of Locality
Local mechanisms allow models to focus on specific image regions rather than processing everything globally. In modern architectures like Vision Transformers and paradigms like self-supervised learning, these mechanisms are essential for making computer vision more efficient, robust to occlusions, and capable of learning from limited data.
The Problem: Limitations of Global Processing
While global image representations have historically dominated deep learning, they face three significant hurdles:
- Computational Waste: Standard ViTs suffer from quadratic complexity; as image resolution increases, memory and processing requirements explode. Processing every pixel equally is inherently inefficient.
- Brittleness: Global models often fail when objects are partially occluded or viewed from unusual angles.
- Data Hunger: Global features are often dataset-specific. Conversely, local features (like textures or specific part shapes) are more transferable, which is critical for tasks with scarce data, such as Few-Shot Learning.
Categorizing Local Mechanisms
The survey divides local mechanisms into three functional eras:
- Hard Part Division: The traditional approach of manually cropping images into fixed grids or patches.
- Soft Part Division: Using attention modules or clustering to automatically identify important regions while suppressing noise.
- The Transformer Era: The current frontier, focusing on CNN hybrids, window-based attention (e.g., Swin Transformers), and token pruning to discard redundant background information.
Self-Supervised Learning and Masked Prediction
One of the most significant shifts in recent years is the move from global contrastive learning to local masked prediction in self-supervised learning (SSL).
As illustrated above, while contrastive learning (a) often relies on global views, Masked Image Modeling (b) forces models to reconstruct missing local patches. This approach has emerged as a powerhouse, demonstrating that predicting local structure is often superior for downstream tasks.
Key Insights: Efficiency and Transferability
- Locality as Inductive Bias: Local mechanisms act as a "shortcut," allowing models to perform better on smaller datasets by assuming nearby pixels are related. Pure ViTs are often "data hungry" because they lack this inherent bias.
- Universal Language: Local features are highly transferable. While a global view of a car and a truck differ, their local parts (wheels, lights) are similar, forming the backbone of Zero-Shot Learning.
- Efficiency via Sparsity: Token pruning can achieve massive speedups (up to 15x in some cases) by identifying and discarding redundant background pixels without sacrificing accuracy.
Future Outlook
The field is moving toward more "knowledgeable" and "selective" models. Future research must address:
- Top-Down Selectivity: Models should dynamically change which local parts they prioritize based on high-level goals.
- Memory Retrieval: AI needs better mechanisms to remember and transfer local knowledge across diverse domains.
- Contextual Reasoning: Balancing local focus with surrounding context remains a challenge; a model must not become so "zoomed in" that it loses vital environmental cues (e.g., recognizing a keyboard is easier when the monitor is visible).
Summary updated