When training deep learning models under weak supervision—such as semi-supervised learning, unsupervised domain adaptation, or learning with noisy labels—we inevitably encounter the trap of confirmation bias.
If a model relies solely on its own prediction confidence (e.g., softmax outputs) to select which pseudo-labels to trust, it tends to memorize and amplify its own mistakes. As Alessandro Conti et al. observe in Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition, raw model confidence is a highly unreliable indicator of accuracy when evaluating out-of-distribution or unlabeled data.
To break this feedback loop, a prominent research direction has emerged: neighborhood-based label refinement. Instead of trusting what a model predicts for an isolated sample, these techniques validate and correct labels by looking at where that sample sits within the latent feature space relative to its neighbors.
Why It Works: The Manifold Hypothesis
The core intuition behind neighborhood refinement relies on a fundamental property of deep neural networks: even when a model's final classification boundary is warped or uncalibrated due to label noise, its internal layers often still learn a coherent, cluster-structured representation space. Under the manifold hypothesis, semantically similar instances naturally cluster together in this high-dimensional embedding space.
Ioannis Patras demonstrates this phenomenon in NoiseBox: Toward More Efficient and Effective Learning With Noisy Labels, showing that a sample's topological consistency with its neighbors provides a far cleaner signal for label correctness than individual loss values or prediction scores. If an instance is labeled "cat" but its closest neighbors in the feature space are all "dog" representations, the local geometry exposes the label as an anomaly.
By shifting the validation mechanism from the model's predictions to the feature space's geometry, researchers can implement a non-parametric sanity check that filters out noise and stabilizes self-training.
Mechanisms of Neighborhood Refinement
Taxonomically, researchers operationalize this local consistency through three primary geometric mechanisms:
1. K-Nearest Neighbor (k-NN) Consensus
The most direct implementation is a local voting scheme. For any given sample, the framework identifies its $k$-nearest neighbors in the feature space and aggregates their labels.
- Error Detection: In Detecting Corrupted Labels Without Training a Model to Predict, Zhaowei Zhu et al. introduce a training-free approach (SimiFeat) that computes a soft label based on an instance's neighborhood distribution. If the observed label deviates significantly from this local consensus, it is flagged as corrupt.
- Candidate Pruning: Shuo He et al. apply this to partial-label learning in Candidate Label Set Pruning: A Data-centric Perspective for Deep Partial-label Learning, using a neighborhood voting mechanism to down-vote and eliminate false candidate labels that lack local topological support.
2. Prototype and Centroid Proximity
Rather than computing individual pairwise distances for every neighbor, models can compute global or batch-level class "prototypes"—the mean feature vector for each category.
[ Class Centroid A ] <--- (Close: Validated as Class A) --- [ Unlabeled Sample ]
|
v (Far from Centroid B)
[ Class Centroid B ]
- Rescuing Moderate Confidence: Standard self-training discards samples with mediocre confidence scores. However, Hui Tang et al. show in Towards Discovering the Effectiveness of Moderately Confident Samples for Semi-Supervised Learning that if a moderately confident sample lies close to its predicted class prototype, it is highly likely to be correct and can be safely "rescued" to guide the decision boundary.
- Label-Specific Denoising: In multi-label text classification, Pengyu Xu et al. develop the LeD framework in Enhancing Multi-Label Text Classification under Label-Dependent Noise: A Label-Specific Denoising Framework, which flags instances as noise if their feature embeddings deviate significantly from the established semantic centroid of their assigned categories.
3. Graph Label Propagation and Bidirectional Walks
When local neighborhoods are chained together, they form a graph that allows labels to flow along high-density data manifolds.
- Graph Cleaning: Tong Wei et al. construct explicit $k$-NN graphs in NGC: A Unified Framework for Learning with Open-World Noisy Data to propagate clean labels across similar data points, effectively "washing" out both in-distribution noise and out-of-distribution outliers.
- Bidirectional Consistency: To enforce an even stricter verification threshold, Junbao Zhuo et al. propose a bidirectional walk metric in Image-to-video Adaptation with Outlier Modeling and Robust Self-learning. A pseudo-label for a target sample is only trusted if its nearest neighbor in the source domain maps back to a sample sharing that same predicted label.
Applications Across Weakly Supervised Regimes
This core mechanism of neighborhood validation has been successfully adapted across several distinct problem domains:
Semi-Supervised Learning with Few Labels
When labeled data is exceptionally scarce, self-training frameworks like NeST, introduced by Ran Xu et al. in Neighborhood-Regularized Self-Training for Learning with Few Labels, use neighborhood divergence metrics to prevent the model from drifting. By ensuring that pseudo-labels match the internal consistency of nearby feature clusters, the model preserves its structural boundaries even when initialized from minimal supervision.
Unsupervised Domain Adaptation (UDA)
In cross-domain tasks, the model must generalize to a target domain without target labels. Yoonhyung Kim et al. show in Semi-Supervised Domain Adaptation via Selective Pseudo Labeling and Progressive Self-Training that target features positioned near labeled source anchors act as "golden samples" that can reliably seed the adaptation process.
Furthermore, Jaehoon Choi et al. demonstrate in Pseudo-Labeling Curriculum for Unsupervised Domain Adaptation that prioritizing pseudo-labels in high-density feature regions early in training prevents the model from being poisoned by the noisy, sparse representations typically found near cross-domain decision boundaries.