Abstract
As the open science movement expands, the volume of shared scientific datasets has grown exponentially. However, retrieving specific datasets remains difficult due to the "vocabulary mismatch problem," where different scientific fields use varying terminology for identical concepts. This paper presents CSDR-BERT, a specialized retrieval model that enhances the Sentence-BERT architecture with unsupervised contrastive learning (SimCSE), a K-Nearest Neighbors (KNN) matching layer, and the Cosent loss function. Evaluated on a custom Chinese scientific dataset and standard benchmarks, CSDR-BERT demonstrates significant improvements in semantic matching accuracy, achieving an F-score of 92.49% on domain-specific tasks.
Introduction
The effective discovery of scientific datasets is a cornerstone of modern research. Despite the proliferation of data repositories, traditional keyword-based search engines often fail to bridge the gap between user queries and dataset metadata. This failure is primarily attributed to the vocabulary mismatch problem: scientific concepts are frequently represented by synonyms (e.g., "chickenpox" vs. "varicella") or distinct notations (e.g., "glucose" vs. "C6H12O6").
While general-purpose pre-trained language models like BERT have advanced natural language processing, they often lack the domain-specific nuances required for specialized scientific retrieval. CSDR-BERT addresses these limitations by refining sentence representations and leveraging local data similarities to improve retrieval precision.
Methodology
CSDR-BERT builds upon the Sentence-BERT (SBERT) framework, introducing three primary architectural enhancements to better capture scientific semantics:
- Unsupervised SimCSE (Contrastive Learning): To produce more robust sentence embeddings, the model employs Simple Contrastive Learning of Sentence Embeddings (SimCSE). By passing the same sentence through the model twice with different dropout masks, the model learns to cluster similar meanings more effectively in the vector space.
- KNN-BERT Matching Layer: Beyond standard linear classification, the authors integrated a K-Nearest Neighbors (KNN) component. The final matching score is a weighted combination of the neural network's output and the KNN result, allowing the model to benefit from local similarities in the training data.
- Cosent Loss Function: The model utilizes Cosine Sentence (Cosent) loss instead of standard cross-entropy. Cosent focuses on the relative similarity between pairs, ensuring that relevant matches are ranked significantly higher than non-matches, which is critical for retrieval performance.
Datasets and Experimental Setup
The model was trained and evaluated using a combination of domain-specific and general-purpose datasets:
- Scientific Dataset: A custom benchmark derived from the National Earth System Science Data Centre (NESDC), containing over 30,000 headings. A subset of 10,960 pairs was manually annotated for similarity.
- Chinese Scientific Literature (CSL): A corpus of approximately 396,000 journal articles. A subset of 10,000 articles was used for unsupervised pre-training via SimCSE.
- Public Benchmarks: To ensure generalizability, the model was tested on standard Chinese NLP tasks, including AFQMC, LCQMC, STS-B, and PAWS-X.
Results and Discussion
Experimental results indicate that CSDR-BERT significantly outperforms baseline models. On the specialized Scientific Dataset, the full model achieved an F-score of 92.49% and an Accuracy of 92.66%, representing a substantial improvement over standard Sentence-BERT configurations.
Ablation studies highlight the contribution of each component. Integrating the KNN layer alone provided a 3.88% boost in accuracy, while the Cosent loss and SimCSE pre-training added 1.45% and 0.49%, respectively.
The model also showed strong cross-domain performance. On the STS-B benchmark, CSDR-BERT reached a Spearman correlation of 80.85%, compared to 70.79% for the baseline. Notably, the model excelled on the PAWS-X dataset, which is designed to challenge models with "word-stacking" (sentences with identical words in different orders), proving its ability to grasp deep semantics rather than just surface-level word overlap.
Conclusion
CSDR-BERT effectively mitigates the vocabulary mismatch problem in Chinese scientific dataset retrieval. By combining contrastive learning with neighborhood-based matching and a ranking-focused loss function, the model provides a robust solution for navigating complex scientific terminology. Future research will focus on incorporating multi-modal metadata—such as temporal and geographical ranges—into the matching process and refining re-ranking algorithms for even higher retrieval precision.
Summary updated