Abstract
Open-set classification (OSC) presents a significant challenge in machine learning, requiring models to accurately categorize known classes while identifying previously unseen novel classes. This task is further complicated in "cold start" scenarios where training data is initially entirely unlabeled. We present ASOCIA (Active Sampling for Open-set Classification without Initial Annotation), a framework that integrates diversity-aware active sampling with latent low-rank representation (LRR). By selecting the most representative and diverse instances for annotation, ASOCIA builds robust subspace-based classifiers with minimal human labeling. Our experiments on benchmark datasets demonstrate that ASOCIA significantly outperforms state-of-the-art methods in both classification and novelty detection.
Introduction
In real-world applications such as face recognition or web content categorization, data often arrives in large, unlabeled streams. Traditional supervised learning, which assumes a closed-set environment, fails when encountering novel categories. Open-set classification (OSC) addresses this, but typically requires substantial labeled data to define the "known" feature space. ASOCIA overcomes this limitation by starting with zero initial labels and utilizing an active sampling strategy to efficiently build an OSC model under a strict labeling budget.
Methodology
The ASOCIA framework operates in two stages: active sample selection and low-rank classification.
Active Selection of Representative Examples
Since no initial labels are available, ASOCIA employs representative sampling rather than uncertainty-based metrics. The model uses a linear reconstruction approach ($X \approx XW$) with two key constraints:
- Structured Sparsity: An $L_{2,1}$ norm constraint on the weight matrix $W$ forces the model to represent the entire dataset using a minimal subset of instances.
- Diversity Constraint: To prevent redundant labeling of similar points, a diversity term minimizes the correlation between selected rows of $W$, ensuring broad coverage of the feature space.
Low-Rank Representation (LRR) for Classification
Once the selected samples are labeled, the model utilizes Latent LRR. This technique assumes that data from the same class resides within the same low-dimensional subspace.
- Affiliation Matrix: The model learns an affiliation matrix $Z$ that captures the relationships between instances.
- Novelty Detection: For test instances, the model calculates a "class mass" score. If an instance does not strongly affiliate with any known class subspaces, it is flagged as novel. An iterative thresholding method is used to optimize the separation between known and novel data.
Experimental Evaluation
We evaluated ASOCIA on three standard image benchmarks: ExtendedYaleB, Fashion-MNIST, and Coil20. To simulate a challenging open-set environment, only 20% of classes were treated as known.
Key Results
ASOCIA consistently outperformed baselines such as ASG and SENC, particularly when the number of labeled examples was small. On the ExtendedYaleB dataset, ASOCIA achieved a known-class accuracy of 93.9% and a novel-class detection accuracy of 99.9%. The framework maintained high F1-scores across all datasets, indicating a balanced performance between classification and novelty detection.
Discussion and Conclusion
The success of ASOCIA highlights the importance of diversity in active sampling for cold-start scenarios. By forcing the selection of diverse representatives, the labeling budget is utilized more effectively. Furthermore, treating classes as subspaces via LRR provides a naturally robust mechanism for OSC, as novel data points are unlikely to align with the specific low-rank structures of known classes.
Future work will focus on extending this framework to handle non-linear data distributions through deep representation learning and addressing hyperparameter sensitivity in the absence of initial labels.
Summary updated