Introduction
Deep clustering aims to learn feature representations and cluster assignments simultaneously without ground-truth labels. A common strategy is pseudo-labeling, where the model treats its own high-confidence predictions as ground truth for subsequent training iterations. However, this approach is susceptible to "error accumulation": if a model incorrectly labels an ambiguous sample (e.g., a husky mistaken for a wolf) and then trains on that label with high confidence, it reinforces its own mistakes in a "garbage in, garbage out" loop.
To address this, the Robust Pseudo-labeling for Semantic Clustering (RPSC) framework introduces a two-stage process designed to filter noise and weight samples based on semantic confidence, preventing the model from over-fitting to early, incorrect guesses.
The RPSC Framework
RPSC operates in two distinct stages: RPSC-Self for initial label generation and RPSC-Semi for iterative refinement.
Stage 1: RPSC-Self and Soft Contrastive Loss
In the first stage, the model utilizes a pre-trained backbone (such as MoCo-v2) to extract initial features. It then calculates cluster centers within the embedding space. The core innovation here is the Soft Contrastive Loss (SCL). Unlike standard contrastive losses that treat all positive pairs equally, SCL incorporates a "semantic confidence" score ($d_{ic}$).
Samples located near cluster centers are assigned high confidence and exert a stronger influence on the model's weight updates. Conversely, samples near cluster boundaries are down-weighted, preventing the model from over-fitting to ambiguous data. Furthermore, SCL intelligently excludes "intra-class" samples from the loss denominator to prevent the accidental repulsion of similar images. To ensure stability, the framework employs multi-head training, selecting the projection head that minimizes the SCL value.
Stage 2: RPSC-Semi and Local Consistency
The second stage refines the initial pseudo-labels by identifying a "gold standard" set of reliable samples. This is achieved through a local-consistency filtering mechanism: a pseudo-label is deemed reliable only if the sample's nearest neighbors in the embedding space share the same label. These reliable samples then guide a semi-supervised learning phase (using the FreeMatch framework), which adaptively adjusts classification thresholds for each category to further sharpen cluster boundaries.
Experimental Results
RPSC was evaluated across six benchmarks, including CIFAR-10, STL-10, and ImageNet subsets. The framework demonstrated significant improvements over 18 baseline algorithms, including SCAN and SPICE.
The most notable performance gain occurred on the ImageNet-Dogs dataset, a challenging benchmark requiring fine-grained discrimination between 15 dog breeds. RPSC achieved an accuracy of 0.688, representing a 24% improvement over previous state-of-the-art methods. On CIFAR-100 (using 20 super-classes), RPSC reached an accuracy of 0.556, outperforming the previous best by nearly 2%.
Ablation studies confirmed that the SCL loss is a versatile component; integrating it into existing frameworks like SCAN and SPICE consistently yielded performance boosts, highlighting its effectiveness in managing boundary-sample noise.
Conclusion and Future Work
While RPSC sets new records on several benchmarks, it faces challenges with high-cardinality datasets like Tiny-ImageNet (200 clusters). In such complex scenarios, if the initial RPSC-Self stage fails to identify enough reliable "anchors," the subsequent refinement stage lacks the necessary foundation for improvement. Future research will explore scaling RPSC to handle larger cluster counts and extending the framework to multi-view clustering and transfer learning applications.
Summary updated