In semi-supervised and unsupervised learning, data is abundant but labels are scarce. A classic solution is pseudo-labeling: training a model on a small set of labeled data, using it to predict labels for unlabeled data, and then adding the most confident predictions back into the training set.
However, this approach suffers from a fundamental flaw: confirmation bias. If a model is highly confident but incorrect about a specific unlabeled sample, it will inject that erroneous "pseudo-label" into its own training pipeline. Over time, the model reinforces its own mistakes, leading to poor generalization.
To break this feedback loop, researchers have shifted away from relying on a single model's internal confidence scores. Instead, they leverage agreement filtering (or consensus-based filtering). The core intuition is simple: while a single model or view can easily hallucinate or overfit to noise, it is statistically highly improbable for multiple independent models, data scales, or feature representations to converge on the exact same incorrect prediction. By requiring a consensus across different perspectives, we can effectively filter out noise and isolate high-fidelity training signals.
Multi-Model Consensus: The Power of Independent Committees
The most direct way to implement agreement filtering is through an ensemble or committee of distinct models. If several models trained with different initializations, architectures, or data splits all agree on an unlabeled sample's label, that sample is highly likely to be correct.
┌───────────┐ ──► Prediction A ┐
│ Model 1 │ │
└───────────┘ ▼
Unlabeled ──► ┌───────────┐ ──► Prediction A ──► Consensus Filter ──► Accept Pseudo-Label
Sample │ Model 2 │ ▲
└───────────┘ │
┌───────────┐ ──► Prediction B ┘
│ Model 3 │
└───────────┘
Dror Simon et al. demonstrated this in Auto-Annotation Quality Prediction for Semi-Supervised Learning with Ensembles, where the degree of agreement across a multi-model ensemble serves as a direct proxy for label correctness. By filtering out pixel-level noise in semantic segmentation tasks based on this consensus, the authors achieved competitive performance using only 30% of the manually labeled data.
This concept extends naturally to iterative co-training frameworks like tri-training. For instance, Kuniaki Saito et al. developed Asymmetric Tri-training for Unsupervised Domain Adaptation, which uses two independent "teacher" networks to cautiously pseudo-label target data for a third "student" network. A pseudo-label is only accepted if both teachers agree on the category and meet a high confidence threshold, preventing the propagation of errors into the student model. Similarly, Zhe Liu in Teacher-Student Learning Paradigm for Tri-training: An Efficient Method for Unlabeled Data Exploitation showed that requiring agreement between independent models before assigning a proxy label significantly reduces error propagation compared to standard self-training.
Agreement filtering even holds true when the models in the committee have vastly different capabilities. In Weak-to-Strong Reasoning, Yuqing Yang et al. showed that a smaller, weaker supervisor model and a larger student model can arrive at the same final answer through independent reasoning paths. This cross-model agreement serves as a robust proxy for truth, allowing the student model to fine-tune on high-confidence, reliable examples while discarding the weaker supervisor's noisy outputs.
Multi-Scale and Multi-View Consistency
Training an ensemble of multiple models can be computationally expensive. An elegant alternative is to generate consensus by passing the same data point through different transformations, spatial scales, or geometric projections within a single model framework.
Spatial and Scale Consensus
In computer vision, an object's identity should remain invariant regardless of the resolution or perspective from which it is viewed.
- Resolution Scales: Hengli Wang et al. introduced this to self-supervised stereo matching in PVStereo: Pyramid Voting Module for End-to-End Self-Supervised Stereo Matching. Their Pyramid Voting Module generates pseudo-labels by comparing disparity maps produced at different image resolutions. Only values that remain stable across these scales are used as supervision, filtering out the noise inherent in traditional non-learning algorithms.
- Geometric Projections: When adapting models to distorted environments, such as $360^\circ$ panoramic views, Xu Zheng et al. proposed 360SFUDA++: Towards Source-free UDA for Panoramic Segmentation by Learning Reliable Category Prototypes. The framework evaluates model confidence by comparing predictions across multiple geometric representations (e.g., equirectangular and tangent projections). Pixels that yield consistent class predictions across these different views are selected as reliable pseudo-labels.
Augmentation Variance
In audio and speech processing, consensus can be found by evaluating a model's stability across different data augmentations. Tiantian Feng et al. leveraged this in Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labeling. Their framework generates multiple augmented versions of an unlabeled audio sample. A pseudo-label is only accepted if the average confidence across all versions is high and the variance between predictions is low, ensuring the model only learns from robust, invariant features.
Cross-Representation and Cross-Task Alignment
Another powerful dimension for agreement filtering is forcing consensus between entirely different feature spaces or distinct but related tasks.
Global vs. Local Features
Clustering errors or representation anomalies often occur when a model looks only at the big picture or only at fine details. However, it rarely makes the exact same error in both spaces simultaneously.
In SECRET: Self-Consistent Pseudo Label Refinement for Unsupervised Domain Adaptive Person Re-identification, Leqi Shen et al. utilized a multi-branch network to extract both global holistic features and local part-based features. They mutually refined pseudo-labels by enforcing consistency between these two spaces:
┌───────────────────┐
┌───►│ Global Features │───► Pseudo-Label X ┐
│ └───────────────────┘ │
Input Image ─┤ ▼
│ ┌───────────────────┐ ├──► Match? Yes ──► Keep
└───►│ Local Features │───► Pseudo-Label X ┘
└───────────────────┘
Instances are discarded during training if their pseudo-labels in the global feature space conflict with the majority labels in the local feature space, wiping out clustering noise.
Cross-Task Validation
Confirmation bias can also be mitigated by validating predictions across models optimized for different, but linguistically or structurally related, tasks. Mian Zhang et al. formalized this in Friend-training: Learning from Models of Different but Related Tasks. By requiring agreement between different task outputs (such as Dialogue Rewriting and Semantic Role Labeling) via a translation matcher, the system filters out pseudo-label noise far more effectively than standard single-model self-training.
Why It Works: The Intuition of Orthogonal Error Distributions
To understand why agreement filtering is so effective, consider the concept of orthogonal error distributions.
When a single model trains on its own pseudo-labels, its errors are highly correlated with its own inductive biases. If the model has a blind spot, it will confidently assign wrong labels to samples in that blind spot, and subsequently freeze those errors into its weights during the next training iteration.
However, when we introduce a second model with a different architecture, a different spatial scale, or a different feature representation, that second model operates with a completely different set of inductive biases. Its blind spots are located in different regions of the data space.
By applying an agreement filter, we are essentially taking the intersection of the correct prediction zones. The models may both make mistakes, but they will rarely make the same mistake on the same unlabeled sample. Consensus filtering acts as a high-pass filter for truth, ensuring that automated supervision pipelines remain clean, stable, and robust.