FNSCC: Fuzzy Neighborhood-Aware Self-Supervised Contrastive Clustering for Short Text

Zijian Zheng, Yonghe Lu, Jian YinEMNLP 2025 · OpenReview

TL;DR

This paper introduces FNSCC, a novel framework for clustering short texts that tackles the challenges of semantic sparsity and fuzzy boundaries by incorporating neighborhood information. FNSCC enhances both instance-level contrastive learning and cluster-level assignments, leading to improved accuracy and consistency in text clustering.

What Problem Does This Paper Address?

Clustering short texts is notoriously difficult. These texts often lack sufficient context, leading to sparse representations and ambiguous category boundaries. While recent contrastive learning methods have improved how individual text snippets are represented, they often overlook the local semantic structure within the clustering process itself. A key issue is that semantically similar texts, which should ideally be grouped together, can be mistakenly treated as negative examples during contrastive learning, hindering the model's ability to distinguish between distinct clusters. This paper aims to bridge this gap by developing a method that leverages neighborhood information to refine both the representation learning and the clustering assignment stages.

Methods

FNSCC builds upon the Self-Supervised Contrastive Clustering (SCCL) framework and introduces two main innovations:

  1. Neighborhood-Aware Negative Sampling (Instance-Level): In contrastive learning, the goal is to pull similar (positive) samples together and push dissimilar (negative) samples apart. FNSCC modifies this by excluding "neighboring" instances from the negative sample set. Neighbors are identified using a k-nearest neighbors (KNN) approach based on cosine similarity. The rationale is that these neighbors are likely to belong to the same cluster, and treating them as negatives would introduce noise and hinder discrimination. By excluding them, FNSCC aims to improve inter-cluster separability.
  2. Fuzzy Neighborhood-Aware Clustering (Cluster-Level): This component refines the soft assignment probabilities of instances to clusters. Instead of relying solely on the distance to cluster centers, FNSCC incorporates information about an instance's neighbors. A "fuzzy neighborhood weighting factor" is introduced, which combines semantic similarity and local density to weigh the influence of neighbors on an instance's cluster assignment. This aims to encourage alignment with semantically coherent clusters and mitigate the impact of noisy single-point assignments.

The framework jointly optimizes these two modules using a combined loss function.

Datasets

The FNSCC algorithm was evaluated on six widely used benchmark short text datasets:

  • AgNews: A subset of 8000 news texts categorized into 4 topics.
  • SearchSnippets: Contains 12340 web search snippets covering 8 different domains.
  • GoogleNews-TS, GoogleNews-T, GoogleNews-S: These are variations of the Google News dataset, comprising 11109 news events across 152 categories, differing in whether they use full texts, titles, or snippets.
  • Tweet: Consists of 2472 tweets distributed across 89 categories.

No new datasets were introduced in this paper.

Key Insights and Ideas

  • Neighborhoods are key for short text clustering: The paper strongly advocates for leveraging local neighborhood information, not just for instance representations but also for refining cluster assignments.
  • Excluding neighbors from negatives is crucial: Treating semantically similar neighbors as negatives is detrimental to clustering performance. FNSCC's approach of excluding them significantly improves inter-cluster separability.
  • Fuzzy neighborhood weighting smooths cluster assignments: Incorporating neighbor information in a "fuzzy" manner (i.e., soft weighting) helps to create more coherent cluster boundaries and increases confidence in soft assignments, especially in areas with ambiguous boundaries.
  • Joint optimization is powerful: Combining neighborhood-aware instance-level contrastive learning with fuzzy neighborhood-aware clustering in an end-to-end framework yields the best results, demonstrating synergy between the two components.

Key Results

FNSCC consistently outperformed state-of-the-art models on most of the benchmark datasets in terms of both Accuracy (ACC) and Normalized Mutual Information (NMI). For instance, on the AgNews dataset, FNSCC achieved an ACC of 87.85 and an NMI of 66.70, which were the highest reported values. While RSTC slightly surpassed FNSCC on a couple of metrics for specific datasets, the overall results validate FNSCC's effectiveness. Ablation studies confirmed that both the instance-level and cluster-level neighborhood-aware mechanisms contribute significantly to the performance gains.

Limitations and Future Work

The authors acknowledge a few limitations:

  • Pre-defined number of clusters: Similar to SCCL, FNSCC requires the number of clusters to be set in advance.
  • Sensitivity to neighbor selection: For datasets with many clusters and imbalanced distributions, FNSCC can be sensitive to the choice of neighbors.

For future work, they plan to:

  • Explore adaptive mechanisms for dynamically determining the optimal number of clusters and neighborhood range, potentially by incorporating prior data knowledge.
  • Investigate methods based on density or local uncertainty to address issues in low-density regions and further refine discrimination strategies.

Summary updated

Abstract

Short texts pose significant challenges for clustering due to semantic sparsity, limited context, and fuzzy category boundaries. Although recent contrastive learning methods improve instance-level representation, they often overlook local semantic structure within the clustering head. Moreover, treating semantically similar neighbors as negatives impair cluster-level discrimination. To address these issues, we propose Fuzzy Neighborhood-Aware Self-Supervised Contrastive Clustering (FNSCC) framework. FNSCC incorporates neighborhood information at both the instance-level and cluster-level. At the instance-level, it excludes neighbors from the negative sample set to enhance inter-cluster separability. At the cluster-level, it introduces fuzzy neighborhood-aware weighting to refine soft assignment probabilities, encouraging alignment with semantically coherent clusters. Experiments on multiple benchmark short text datasets demonstrate that FNSCC consistently outperforms state-of-the-art models in accuracy and normalized mutual information. Our code is available at https://github.com/zjzone/FNSCC.
BibTeX
@inproceedings{zheng2025fnscc,
  author = {Zijian Zheng and Yonghe Lu and Jian Yin},
  title = {{FNSCC: Fuzzy Neighborhood-Aware Self-Supervised Contrastive Clustering for Short Text}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=CXvGjmMDXJ},
}
Figure