Introduction
Most semi-supervised learning methods operate under a closed-world assumption: they presume we know every category the model will ever encounter. However, real-world data is rarely so tidy. This paper introduces CWAEE (Class-Wise Adaptive Exploration and Exploitation), a novel framework designed to handle the messy reality where unlabeled training data contains unknown classes. By leveraging adaptive thresholds and Fourier-based data augmentation, CWAEE successfully generalizes known classes to entirely new domains while accurately flagging unknown ones.
The Open-Set Domain Generalization Problem
In traditional machine learning, models rely on the "i.i.d. assumption"—that training and testing data are drawn from the exact same distribution. Domain Generalization (DG) challenges this by requiring a model to learn from several distinct "source" domains (e.g., photos and cartoons) and perform accurately on a completely unseen "target" domain (e.g., sketches).
Semi-Supervised Domain Generalization (SSDG) makes this setup more practical by utilizing a small set of labeled samples alongside a massive pool of unlabeled data. Unfortunately, existing SSDG methods have a critical blind spot: they assume every unlabeled image belongs to one of the "known" classes present in the labeled set.
In realistic scenarios, unlabeled data is "open-set." It contains "unknown" classes that the model has never been trained to recognize. If a model is forced to assign a known label (like "dog") to an unlabeled image of an unknown class (like "guitar"), it suffers from negative transfer, severely degrading its performance on actual known classes.
The CWAEE Methodology
To tackle this open-set challenge, the authors propose CWAEE, which divides the learning process into two distinct phases: Exploration and Exploitation.
1. Exploration (Detecting Unknowns)
- One-vs-Rest Classifiers: Instead of relying on a standard "winner-takes-all" softmax classifier—which is notoriously overconfident on out-of-distribution data—CWAEE employs one-vs-rest classifiers using sigmoid functions. This structural change allows the model to effectively say "I don't know" when an image doesn't fit any known category.
- Adaptive Thresholds via BMM: The framework abandons fixed confidence thresholds (e.g., a rigid 0.9 cutoff). Instead, it uses a Beta Mixture Model (BMM) to dynamically learn class-specific thresholds. This adapts to the reality that some classes are inherently harder to identify than others, allowing the model to decide what "confident" looks like for each specific category.
2. Exploitation (Improving Generalization)
- Fourier Transformation for Style Swapping: To build robustness against domain shifts, CWAEE utilizes a clever Fourier transform technique. It swaps the "amplitude" spectrum (which encodes the "style" or domain characteristics of an image, like brushstrokes or lighting) between different samples while preserving the "phase" spectrum (which retains the "semantics" or actual object shape).
- Consistency Regularization: The model is then trained to output the same prediction for an original image and its style-swapped counterpart. This consistency forces the network to ignore superficial domain noise and focus on high-level, generalizable semantic features.
Datasets and Evaluation
The researchers evaluated CWAEE on three standard computer vision benchmarks, specifically modified to include "seen unknown" (in unlabeled source data) and "unseen unknown" (in target data) classes:
- PACS: A diverse dataset containing Photos, Art, Cartoons, and Sketches.
- Office-Home: A complex dataset with 65 classes across Art, Clipart, Product, and Real-World domains.
- miniDomainNet: A large-scale subset of DomainNet featuring 126 classes.
Key Results
CWAEE consistently outperformed state-of-the-art methods, such as FixMatch and StyleMatch, across all evaluated datasets.
- Accuracy on Known Classes: On the PACS dataset, CWAEE achieved an average accuracy of 84.20%, significantly outpacing the next best competitor (StyleMatch at 80.69%).
- Detecting Unknowns (AUROC): The method demonstrated a massive leap in its ability to identify unknown classes. On PACS, it reached an AUROC of 79.21%, whereas most baseline methods struggled between 50% and 60%.
- Robustness to Unknowns: As shown in the results, CWAEE maintains high accuracy and AUROC even as the number of "seen unknown" classes in the unlabeled data increases, proving its resilience in highly noisy environments.
Limitations and Future Directions
While powerful, CWAEE has a few limitations. It relies on a small validation set of known classes to calibrate temperature scaling for the classifiers—data that might be unavailable in the strictest domain generalization settings. Additionally, maintaining score queues and fitting Beta Mixture Models for every class introduces computational overhead compared to simpler SSDG approaches.
Looking forward, the authors suggest that this adaptive exploration logic could be highly valuable in safety-critical fields like medical diagnosis or autonomous driving, where accurately identifying "unknown" scenarios is just as important as classifying known ones.
Summary updated