Abstract
Positive-Unlabeled (PU) learning is a common challenge in real-world machine learning where only a subset of positive instances is labeled, and the remaining data is unlabeled. In high-dimensional spaces, such as image datasets, positive and negative distributions often become deeply entangled, making traditional classification difficult. We propose a novel representation learning method, VQ K-means, which forces unlabeled data into two distinct, easily separable clusters. By combining a specialized "tug-of-war" loss function with vector quantization, our approach simplifies high-dimensional PU learning into a manageable clustering problem. Experimental results on benchmarks like AFHQ demonstrate that our method significantly outperforms current state-of-the-art models in both accuracy and convergence speed.
Introduction
In many practical scenarios—ranging from medical diagnosis to fraud detection—confirmed negative labels are unavailable. We typically possess a set of "positive" examples and a large pool of "unlabeled" data that contains both hidden positives and negatives. This is the core of PU Learning. While low-dimensional PU problems can often be solved by identifying distinct modes in a distribution, high-dimensional data (like images) lacks this inherent simplicity. Current deep learning methods often struggle to find a clear decision boundary without explicit negative supervision, leading to slow training and suboptimal performance.
Methodology: VQ K-means
Our approach focuses on learning a new representation of the data that mimics the simplicity of low-dimensional toy problems where K-means clustering is effective.
The "Tug-of-War" Loss
The central innovation is a loss function designed to create distributional contrast. We define two target vectors in the representation space: a "high-magnitude" vector for labeled positives and a "low-magnitude" vector for all unlabeled samples.
- Labeled Positives: Pulled toward the high-magnitude target.
- Unlabeled Samples: Pulled toward the low-magnitude target.
Because the unlabeled set contains both positives and negatives, the unlabeled positive samples experience a "tug-of-war" between the two targets, eventually settling into a linear combination. Unlabeled negatives, however, are only pulled toward the low-magnitude target. This mechanism naturally separates the two classes into distinct clusters.
Vector Quantization (VQ)
To prevent these clusters from overlapping and to sharpen the decision boundary, we incorporate vector quantization. As shown in the architecture diagram below, the learned representations are forced to "snap" to specific codebook vectors.
Stopping Criterion
A major hurdle in PU learning is the lack of a labeled validation set. We observed that as the model begins to overfit, the centers of the learned clusters start to converge. By monitoring the distance between these cluster centers, we can identify the optimal stopping point—the peak of accuracy—without needing any negative labels for validation.
Experimental Results
We evaluated VQ K-means on several image benchmarks, including MNIST, Fashion-MNIST, CIFAR-10, and AFHQ (Animal Faces).
Disentanglement and Accuracy
The primary goal was to achieve clear disentanglement in high-dimensional space. On the AFHQ dataset (Cats vs. Dogs), our method achieved a state-of-the-art accuracy of 95.3%, surpassing baselines like Observer-GAN and TEDn. The effectiveness of the disentanglement is clearly visible in t-SNE projections of the learned space.
Training Efficiency
Beyond accuracy, VQ K-means offers a dramatic improvement in training speed. On the AFHQ dataset, our model reached 90% of its peak accuracy in just 11 epochs. In contrast, competing methods required between 43 and 220 epochs to reach similar performance levels.
Conclusion
By reframing high-dimensional PU learning as a representation learning problem, VQ K-means provides a robust and efficient solution. The combination of a contrastive loss and vector quantization successfully "disentangles" positive and negative distributions, making them as separable as a simple 1D toy problem. While the method currently relies on the "Selected Completely At Random" (SCAR) assumption, its superior speed and accuracy make it a powerful tool for real-world unlabeled datasets.
Summary updated