The Seed Dependency Problem in Semi-Supervised Learning

This direction is split into other directions: seed data dependency. Its last version stays readable and citable here.

Version 1 of 1124 papers127 conceptsfrozen 2026-10-04

Semi-supervised learning (SSL) promises a world where machines learn from the vast, uncurated wilderness of unlabeled data, requiring only a tiny fraction of the expensive, human-annotated "truth" used in traditional supervised learning. However, across domains ranging from speech recognition to medical imaging, researchers have encountered a persistent structural barrier: the Seed Dependency.

This dependency is a "cold start" problem. To leverage unlabeled data, a model must first have a sufficiently accurate internal representation to make sense of it. If the initial "seed" of labeled data is too small or of poor quality, the model’s first guesses (pseudo-labels) will be wrong, leading to a feedback loop of error propagation that prevents convergence.

The Initialization Bottleneck

The core mechanism of most semi-supervised systems is an iterative cycle: a model trained on a small labeled set predicts labels for a large unlabeled set, and then retrains itself on both. As Keith C. C. Chan notes, this initial labeling phase remains a resource-intensive bottleneck.

Without a strong starting point, the "iterative improvement cycle" described by Hynek Hermansky and Samuel Thomas 0001 in Cross-lingual and multi-stream posterior features for low resource LVCSR systems cannot effectively begin. If the initial model is weak, the resulting low-quality labels act as "noise" rather than "signal," poisoning the training process from the outset.

The Risk of Gradual Drift and Degeneracy

Why can't we just start with one or two examples? De-Chuan Zhan and Zhi-Hua Zhou explored this extreme in Semi-Supervised Learning with Very Few Labeled Training Examples, noting that a minimum threshold is required to establish even a basic decision boundary.

When this threshold isn't met, models suffer from two primary failures:

  1. Gradual Drift: As identified by Xuming Hu and Philip S. Yu in Semi-supervised Relation Extraction via Incremental Meta Self-Training, the model’s predictions slowly move away from the true distribution because it is essentially "learning from its own mistakes."
  2. Degenerate Solutions: Igor Labutov and Tom M. Mitchell found that without a high-quality seed, joint models often fail to establish any meaningful correspondence between inputs and logical forms, collapsing into trivial or useless predictions.

Strategies to Lower the Seed Threshold

To combat this, researchers have developed "warm-up" strategies and alternative forms of supervision that are cheaper than full manual labeling but more robust than pure guessing.

Curriculum and Auxiliary Tasks

One approach is to make the initial learning task easier. Hoel Kervadec and Ismail Ben Ayed in Curriculum Semi-supervised Segmentation suggest teaching a model a simpler auxiliary task—like predicting the size of an organ—before asking it to perform pixel-by-pixel segmentation. This provides a "stable baseline" that makes the subsequent semi-supervised phase more reliable.

Point and Keyword Anchors

Instead of full bounding boxes or transcriptions, researchers are using "cheap" seeds. Yongtao Ge introduced Point-Teaching, which uses single-click point annotations to anchor object detection. Similarly, Weijie Xu and colleagues use "seed keywords" in vONTSS to align neural topic models with human intuition. These anchors serve as the "ground truth" necessary to prevent the model from drifting during the unsupervised phase.

Biased Self-Supervision

In speech recognition, Florian Kreyssig and Abdelrahman Mohamed demonstrated in Biased Self-supervised Learning for ASR that injecting even a tiny amount of labeled data directly into the self-supervised pre-training loop—rather than just at the end—can lead to significantly faster convergence and better performance.

The Frontier of Zero Resource Initialization

The ultimate goal remains a "purely zero-resource environment." As Junxiao Shen and Evan Strasnick argue, transitioning to fully unsupervised approaches requires eliminating the need for any human intervention in the initialization phase.

Current research, such as the work by Jiangmeng Li on Evolutionary Game Theory, is attempting to find mathematical equilibrium points that can replace human-labeled seeds. Until then, the "seed" remains the essential spark that allows semi-supervised models to turn the "noise" of unlabeled data into the "signal" of learned intelligence.

Papers in this version

Showing 60 of 124 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_the_seed_dependency_problem_in_semi_supe_d1dec3bf,
  title = {The Seed Dependency Problem in Semi-Supervised Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/the-seed-dependency-problem-in-semi-supervised-learning/rdr_d1dec3bf6eac4cdf9e3b8901106fdbd5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_the_seed_dependency_problem_in_semi_supe_d1dec3bf_v1,
  title = {The Seed Dependency Problem in Semi-Supervised Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/the-seed-dependency-problem-in-semi-supervised-learning/rdr_d1dec3bf6eac4cdf9e3b8901106fdbd5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}