Abstract
Deep clustering models often struggle with error propagation caused by learning from noisy, self-generated "pseudo-labels." This paper introduces DCGLU (Deep Clustering optimization from Generalized Labeled and Unlabeled data), a plug-and-play optimization method that treats low-confidence samples as unlabeled data. By applying Positive-Unlabeled (PU) learning principles, DCGLU breaks the cycle of error propagation, significantly boosting text clustering performance with negligible computational overhead.
Introduction
Deep text clustering aims to simultaneously learn feature representations and data groupings. In the absence of ground-truth labels, models rely on "generalized labels"—weak supervision signals derived from cluster assignments or sample similarities. However, these labels are inherently noisy, particularly in early training stages. Learning from erroneous labels leads to poor feature representations, which further degrades label quality. This feedback loop, known as error propagation, is a significant bottleneck for state-of-the-art systems.
The DCGLU Framework
The proposed DCGLU method addresses this by relaxing the requirement that every sample must have a definitive pseudo-label. Instead of forcing the model to trust every noisy assignment, DCGLU splits samples into high-confidence and low-confidence pools based on a confidence threshold $t$. High-confidence samples are treated as labeled (Positive), while low-confidence samples are treated as unlabeled.
The core of DCGLU is an adaptation of Positive-Unlabeled (PU) learning for multi-class clustering. By using a non-negative risk estimator, the model learns from the distribution of unlabeled data without being misled by specific noisy labels. DCGLU acts as a regularization term that can be integrated into existing deep clustering loss functions:
$$\mathcal{L}_{total} = \mathcal{L}_{original} + \lambda \cdot \hat{R}_{pu}(f)$$
Experimental Evaluation
The authors evaluated DCGLU on two widely used text benchmarks: SNIPS (a personal voice assistant dataset with 7 intent categories) and DBPedia (a subset of the DBPedia ontology with 14 classes). They compared DCGLU against a suite of baselines ranging from classical K-Means to modern deep systems like Deep Adaptive Clustering (DAC) and Deep Dirichlet Process Mixture (DeepDPM).
Key Results
Experimental results demonstrate that DCGLU consistently improves performance across all metrics, including Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and Accuracy (ACC).
Key findings include:
- State-of-the-Art Boosts: Adding DCGLU to the DeepDPM model pushed its accuracy on the SNIPS dataset from 88.35% to 92.07%. On DBPedia, DCGLU improved the DAC model's accuracy by nearly 4 percentage points (from 63.27% to 66.90%).
- Noise Mitigation: Low-confidence samples are the primary source of feature space corruption. Treating them as unlabeled preserves their structural information without introducing label noise.
- Universality and Efficiency: DCGLU is model-agnostic and improved every architecture tested. It scales linearly ($O(N)$) and typically adds less than 6% to total training time.
Conclusion and Future Work
DCGLU provides a robust, efficient solution to the error propagation problem in deep clustering by leveraging PU learning. While this study focused on text clustering, the authors plan to extend the framework to image and audio data, and explore automated methods for dynamic confidence threshold estimation.
Summary updated