Introduction
In standard Semi-Supervised Learning (SSL), models typically assume that all unlabeled data belongs to a set of known "In-Distribution" (ID) classes. However, real-world unlabeled pools are often contaminated with "Out-of-Distribution" (OOD) samples from classes the model has never encountered. This scenario, known as Open-Set Semi-Supervised Learning (OSSL), presents a significant challenge: how to leverage unlabeled data without being misled by novel OOD samples.
Existing OSSL methods often fall into two extremes. Aggressive approaches attempt to force-align OOD samples into a synthetic "catch-all" class, which often leads to "geometric collapse"—where the model overfits to specific OOD examples and fails to generalize to unseen OOD types. Conversely, conservative methods simply discard low-confidence samples, losing valuable geometric cues that could help define the boundaries of the ID space. SkipAlign proposes a middle ground through a principle called Selective Non-Alignment (SNA).
Methodology: Selective Non-Alignment
The core of the SkipAlign framework is treating the embedding space like a universe of compact ID "galaxies" surrounded by a vast OOD "interstellar void." While traditional contrastive learning relies on "Pulling" (attracting similar samples) and "Pushing" (repelling dissimilar ones), SNA introduces a third operator: Skip.
Dual-Gate ID Selection
To determine which samples to "pull," SkipAlign employs a Dual-Gate selection mechanism. A sample must pass high-confidence thresholds from both a standard ID classifier and a specialized One-vs-All (OVA) OOD detector. Only these "confident ID" samples are pulled toward class prototypes.
The "Skip" Operator and Angular Updates
For samples that fail the dual-gate check—including both seen OOD and uncertain ID samples—the model skips the pull operation. Instead, it applies a non-selective angular update. This provides a "torque" that rotates uncertain features away from ID prototypes without pinning them to a specific synthetic class. This prevents geometric collapse and allows the OOD space to remain naturally heterogeneous.
Key Insights: Norm as a Signal
A striking consequence of the SNA strategy is the emergence of the feature norm as a powerful indicator for OOD detection. Because the model only "pulls" (and thus increases the feature norm of) confident ID samples, these samples naturally form dense, high-norm clusters. In contrast, OOD samples remain dispersed in a low-norm, sparse region of the feature space.
As shown in the empirical visualizations above, ID classes (like Bird, Cat, Dog) exhibit significantly higher feature norms compared to both seen and unseen OOD samples. This separation allows the model to naturally spot anything that does not resemble a known "galaxy."
Experimental Results
SkipAlign was evaluated on standard benchmarks including CIFAR-10, CIFAR-100, and ImageNet-30, specifically testing for unseen OOD generalization. The method consistently outperformed state-of-the-art baselines like SCOMatch and ProSub:
- OOD Detection: SkipAlign improved the Overall-OOD AUC by an average of 3.1 points, with gains reaching up to 7.1 points in difficult scenarios.
- ID Accuracy: These gains in OOD detection did not compromise classification performance; the model maintained or slightly improved closed-set accuracy across datasets.
- Robustness: The framework proved equally effective at detecting "seen" OOD (classes present in the unlabeled training pool) and "unseen" OOD (entirely new classes encountered only at test time).
Conclusion and Future Work
SkipAlign demonstrates that by "letting the void be void," models can achieve superior robustness in open-set environments. By avoiding the forced alignment of uncertain samples, the framework preserves the natural geometry of the feature space. Future research may focus on improving detection for OOD classes that have high semantic overlap with ID classes and further exploring how different network layers contribute to the SNA signal.
Summary updated