Abstract
Descriptive document clustering aims to provide human-readable labels for automatically generated groups. However, existing methods often rely on subjective heuristics and manual hyperparameter tuning. We present the Predictive-Descriptive Clustering (PDC) framework, a self-tuned system that treats clustering as a dual prediction task. By decoupling the cluster-to-feature decoder and the feature-to-cluster encoder, PDC automatically determines the optimal number of clusters and selects concise, positively correlated descriptive keywords. Experimental results across diverse datasets demonstrate that PDC achieves superior interpretability and accuracy compared to state-of-the-art baselines.
Introduction
Clustering is a fundamental tool in exploratory data analysis, yet it often functions as a "black box," providing groups of data without clear explanations of their contents. Descriptive clustering seeks to bridge this gap by generating concise summaries, typically keyword lists, for each cluster.
Two primary challenges hinder the effectiveness of current descriptive clustering techniques. First, there is a lack of objective evaluation; labeling quality is often judged by human intuition rather than rigorous quantitative metrics. Second, hyperparameter selection—such as the number of clusters ($C$) or the number of descriptive features—is usually left to user guesswork. The Predictive-Descriptive Clustering (PDC) framework addresses these issues by framing clustering as a two-way prediction problem within a self-tuning architecture.
The Predictive-Descriptive Clustering (PDC) Framework
The PDC framework models clustering as a decoupled auto-encoder network where clusters must predict features, and a subset of features must predict the clusters. The notation used throughout the framework is summarized in the table below.
Architecture and Process
The PDC process is divided into two main components: the Decoder and the Encoder, as illustrated in Figure 1.
- The Decoder (Cluster $\to$ Features): This component evaluates how well a cluster assignment $\mathbf{y}$ can predict the occurrence of features $\mathbf{x}$. By using multinomial or Bernoulli regression and the Akaike Information Criterion (AIC), the system automatically identifies the optimal number of clusters $C$ that best explains the data variance without overfitting.
- The Encoder (Features $\to$ Cluster): This component identifies a sparse subset of features $\mathcal{S}^{(c)}$ that can accurately predict the cluster assignment $\hat{\mathbf{y}}$. This subset serves as the human-readable description of the cluster.
Key Innovations
- Positivity Constraint: Unlike standard classifiers that might use the absence of a feature (e.g., "not-movie") to define a cluster, PDC enforces a positivity constraint. This ensures that descriptions are based on features present in the cluster, aligning machine logic with human interpretability.
- Self-Tuning via BIC: The model uses the Bayesian Information Criterion (BIC) to balance the trade-off between description accuracy and brevity. This allows the system to automatically determine the "just right" number of keywords for each cluster description.
- Decoupled Efficiency: By separating the encoder and decoder, PDC can efficiently evaluate various clustering configurations without the need for full retraining, significantly reducing computational overhead.
Experimental Evaluation
The authors validated the PDC framework across several diverse text datasets, including 20-Newsgroups, Amazon Reviews (spanning multiple categories), Reuters-21578, Yummly Recipes, and NSF Abstracts.
Performance Metrics
PDC, specifically utilizing the CMIM+ algorithm for feature selection, consistently outperformed existing topic-labeling methods like MSZ. Performance was measured using $F_1$-score and Normalized Mutual Information (NMI), where PDC showed superior alignment with ground-truth labels.
Interpretability and Scaling
Qualitative analysis confirmed that the selected keywords were highly descriptive. For instance, a cluster of kitchen product reviews yielded keywords such as cuisinart, dishwasher, blender, and skillet (see Figure 1). Furthermore, the AIC-based selection for the number of clusters closely matched "oracle" selections, demonstrating the system's robust self-tuning capabilities.
Conclusion
The PDC framework transforms descriptive clustering from a subjective heuristic task into an objective predictive one. By enforcing positivity and utilizing information-theoretic criteria for self-tuning, it provides a scalable and interpretable solution for document organization. Future work may extend this framework to handle complex features like n-grams, hierarchical cluster structures, and non-textual sparse data such as genomic sequences or consumer purchase histories.
Summary updated