Deep learning typically relies on massive, manually labeled datasets to achieve high performance. However, in many specialized domains—ranging from historical archive analysis to high-speed train engineering—acquiring such data is prohibitively expensive or physically impossible. This "annotation tax," as described by Maxime Masson et al. in Stratégies optimales pour l'analyse multidimensionnelle de contenus multilingues issus des réseaux sociaux, creates a bottleneck for AI deployment. Few-shot learning (FSL) addresses this by enabling models to generalize to new tasks using only a handful of examples, often between one and fifty.
Learning to Learn via Meta-Learning
The core intuition behind many few-shot systems is that the model should not just learn a specific task, but rather learn the process of learning itself. This is often framed as meta-learning. Kwonjoon Lee et al. proposed MetaOptNet in Meta-Learning With Differentiable Convex Optimization., which uses high-margin classifiers like Support Vector Machines (SVMs) as the internal "learners." By making the optimization of these classifiers differentiable, the authors allow the global model to learn feature embeddings that are specifically structured to be easily separable by simple linear classifiers, even when data is scarce.
In the context of object detection, this paradigm shifts toward Low-Shot Object Detection (LSOD). As surveyed by Qihan Huang et al. in A Survey of Deep Learning for Low-Shot Object Detection, these methods often involve pre-training on abundant "base" classes and then fine-tuning or using meta-learning to adapt to "novel" classes with minimal supervision.
Leveraging Foundation Models and Prompting
A significant shift in FSL involves using large, pre-trained "foundation models" as a starting point. Instead of extensive retraining, researchers use specialized techniques to "nudge" these models toward new tasks. Maxime Masson et al. demonstrate in Optimal strategies to perform multilingual analysis of social content for a novel dataset in the tourism domain that prompt-based techniques like PET and EntLM can achieve high accuracy in sentiment and entity extraction with as few as 10 labeled examples.
For vision-language models like CLIP, adaptation requires managing new information without overwriting existing knowledge. Yabin Zhang et al. introduced Dual Memory Networks (DMN) in Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models. Their approach uses a dynamic memory cache to store features from test samples online, allowing the model to adapt to new distributions without gradient-based training.
Data Augmentation and Structural Priors
When real data is missing, synthetic data can fill the gap. Haixing Dai et al. developed AugGPT, described in AugGPT: Leveraging ChatGPT for Text Data Augmentation, which uses large language models to generate diverse, semantically similar paraphrases. This increases the effective size of the training set in few-shot scenarios, leading to more robust classifiers.
Beyond text, structural priors help models "guess" missing information. In Inferring and Leveraging Parts from Object Shape for Improving Semantic Image Synthesis, Yuxiang Wei et al. show that few-shot learning can be used to predict the internal parts of an object (like the wheels of a car) from just a silhouette. By learning these structural relationships from a few reference examples, the model can synthesize realistic images of novel categories without needing massive part-level datasets.
Applications in Resource-Constrained Domains
The practical value of FSL is most evident in specialized scientific and engineering fields:
- Remote Sensing: Ilham Adi Panuntun et al. utilized FSL in LinkNet-Spectral-Spatial-Temporal Transformer Based on Few-Shot Learning for Mangrove Loss Detection with Small Dataset to detect environmental degradation where satellite ground-truth data is limited.
- Engineering: Shaodi Dong and Tengfei Jing applied FSL to optimize the crashworthiness of high-speed train structures, where physical crash tests are prohibitively expensive (A Few-Shot Learning-Based Crashworthiness Analysis and Optimization for Multi-Cell Structure of High-Speed Train).
- Software Testing: In FlakyCat: Predicting Flaky Tests Categories using Few-Shot Learning, Amal Akli et al. used CodeBERT and FSL to categorize "flaky" software tests, helping developers identify root causes even when specific types of test failures are rare.
By combining meta-learning strategies, the generative power of foundation models, and domain-specific structural priors, few-shot learning transforms AI from a "big data" tool into a flexible system capable of operating in data-scarce environments.
Key Foundational Work
Meta-Learning With Differentiable Convex Optimization. (2019) — contributed 1 concept elements to this direction.
Few-Shot Few-Shot Learning and the role of Spatial Attention (2020) — contributed 1 concept elements to this direction.
Flight of the PEGASUS? Comparing Transformers on Few-shot and Zero-shot Multi-document Abstractive Summarization (2020) — contributed 1 concept elements to this direction.