Generalizing from Limited Examples with Few-Shot Learning

45 concepts44 papers1 proposal

Deep learning typically relies on massive, manually labeled datasets to achieve high performance. However, in many specialized domains—ranging from historical archive analysis to high-speed train engineering—acquiring such data is prohibitively expensive or physically impossible. This "annotation tax," as described by Maxime Masson et al. in Stratégies optimales pour l'analyse multidimensionnelle de contenus multilingues issus des réseaux sociaux, creates a bottleneck for AI deployment. Few-shot learning (FSL) addresses this by enabling models to generalize to new tasks using only a handful of examples, often between one and fifty.

Learning to Learn via Meta-Learning

The core intuition behind many few-shot systems is that the model should not just learn a specific task, but rather learn the process of learning itself. This is often framed as meta-learning. Kwonjoon Lee et al. proposed MetaOptNet in Meta-Learning With Differentiable Convex Optimization., which uses high-margin classifiers like Support Vector Machines (SVMs) as the internal "learners." By making the optimization of these classifiers differentiable, the authors allow the global model to learn feature embeddings that are specifically structured to be easily separable by simple linear classifiers, even when data is scarce.

In the context of object detection, this paradigm shifts toward Low-Shot Object Detection (LSOD). As surveyed by Qihan Huang et al. in A Survey of Deep Learning for Low-Shot Object Detection, these methods often involve pre-training on abundant "base" classes and then fine-tuning or using meta-learning to adapt to "novel" classes with minimal supervision.

Leveraging Foundation Models and Prompting

A significant shift in FSL involves using large, pre-trained "foundation models" as a starting point. Instead of extensive retraining, researchers use specialized techniques to "nudge" these models toward new tasks. Maxime Masson et al. demonstrate in Optimal strategies to perform multilingual analysis of social content for a novel dataset in the tourism domain that prompt-based techniques like PET and EntLM can achieve high accuracy in sentiment and entity extraction with as few as 10 labeled examples.

For vision-language models like CLIP, adaptation requires managing new information without overwriting existing knowledge. Yabin Zhang et al. introduced Dual Memory Networks (DMN) in Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models. Their approach uses a dynamic memory cache to store features from test samples online, allowing the model to adapt to new distributions without gradient-based training.

Data Augmentation and Structural Priors

When real data is missing, synthetic data can fill the gap. Haixing Dai et al. developed AugGPT, described in AugGPT: Leveraging ChatGPT for Text Data Augmentation, which uses large language models to generate diverse, semantically similar paraphrases. This increases the effective size of the training set in few-shot scenarios, leading to more robust classifiers.

Beyond text, structural priors help models "guess" missing information. In Inferring and Leveraging Parts from Object Shape for Improving Semantic Image Synthesis, Yuxiang Wei et al. show that few-shot learning can be used to predict the internal parts of an object (like the wheels of a car) from just a silhouette. By learning these structural relationships from a few reference examples, the model can synthesize realistic images of novel categories without needing massive part-level datasets.

Applications in Resource-Constrained Domains

The practical value of FSL is most evident in specialized scientific and engineering fields:

By combining meta-learning strategies, the generative power of foundation models, and domain-specific structural priors, few-shot learning transforms AI from a "big data" tool into a flexible system capable of operating in data-scarce environments.

Key Foundational Work

Meta-Learning With Differentiable Convex Optimization. (2019) — contributed 1 concept elements to this direction.
Few-Shot Few-Shot Learning and the role of Spatial Attention (2020) — contributed 1 concept elements to this direction.
Flight of the PEGASUS? Comparing Transformers on Few-shot and Zero-shot Multi-document Abstractive Summarization (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_few_shot_learning_9daef7c9,
  title = {few-shot learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/few-shot-learning/rdr_9daef7c990de4abdb053a294ffaf5bfc},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_few_shot_learning_9daef7c9_v1,
  title = {few-shot learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/few-shot-learning/rdr_9daef7c990de4abdb053a294ffaf5bfc/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake