Abstract
Taxonomy completion, a self-supervised task, aims to add new concepts to an existing taxonomy by attaching them to appropriate hypernym and hyponym pairs. Researchers have proposed several approaches to capture the essential relationships in taxonomy using semantic or structural information. However, they either construct training signals from a single view or simply use a random sampling strategy, making it insufficient to capture various relations in taxonomic structure and learn quality representations. To address this, we propose CoSTC, a contrastive learning framework that captures diverse relations and improves representations for taxonomy completion. It uses two contrasting views, namely intra-view and inter-view, to provide rich self-supervised signals. In intra-view contrasting, we exploit the correlations within queries and within positions by performing instance-level discrimination task. In inter-view contrasting, we use a sampling strategy that considers diversity and hardness to select representative pairs, enhancing the learning of fine-grained query-position relations. Experimental results on three datasets verify the effectiveness of our approach.
Introduction
Taxonomies—hierarchical structures like "Fruit → Apple → Granny Smith"—are essential for search engines, recommendation systems, and medical indexing. However, as new concepts emerge (e.g., "Prompt Engineering" in Computer Science), manually updating these trees is labor-intensive. The task of Taxonomy Completion (TC) involves taking a new "query" concept and finding its rightful place between an existing hypernym (parent) and hyponym (child).
Current methods struggle because they often rely on single-view signals, ignoring broader structural correlations, and use weak training signals. Most models use random sampling to pick "wrong" answers (negatives) for training. If the "wrong" answer is too obvious, the model fails to learn the fine-grained nuances required to distinguish between very similar branches of a tree.
The CoSTC Framework
We introduce CoSTC (Contrastive Self-supervised Taxonomy Completion), which operates in two main phases to capture both semantic and structural relationships.
1. Intra-view Contrastive Pre-training
Before matching queries to positions, the model performs "self-discrimination." It learns to distinguish a concept from all other concepts (query-query) and a position from all other positions (position-position). This phase uses PromptBERT to turn text definitions into high-quality embeddings, ensuring the model understands the semantic "neighborhood" of a concept before the matching begins.
2. Inter-view Contrastive Matching
This is the core matching task, refined by a novel Negative Mining Strategy:
- Topological Diversity Sampling: Instead of picking random negatives, the model selects "wrong" positions from different distances in the tree (local vs. global). This ensures the model understands the hierarchy's overall shape.
- Hard Negative Sampling: The model specifically hunts for "hard" negatives—positions that are semantically very close to the correct one but are technically incorrect.
- Curriculum Learning: Training starts with "easy" negatives and gradually introduces "harder" ones by dynamically adjusting the margin in the loss function, preventing early training instability.
Experimental Results
CoSTC was evaluated on three benchmarks: SemEval-Food, MeSH (Medical Subject Headings), and WordNet-Verb. The model consistently outperformed all baselines, including TMN, QEN, and TaxoComplete.
- Accuracy Gains: On the SemEval-Food dataset, CoSTC achieved a Mean Reciprocal Rank (MRR) of 0.658, a significant jump from the previous state-of-the-art (0.489).
- Recall: It improved Recall@10 by 16.3% on Food, 6.4% on MeSH, and 8.2% on WordNet-Verb compared to the second-best models.
- Ablation Studies: Removing the pre-training or the sampling strategy led to significant performance drops, proving that the dual-view contrastive framework is essential for high-quality representation learning.
Conclusion
CoSTC addresses the limitations of single-view taxonomy completion by leveraging dual-view contrastive signals and sophisticated negative mining. While the model currently relies on text definitions and assumes a static existing hierarchy, future work will explore modeling inter-query relationships and handling potential errors in the initial taxonomy structure.
Summary updated