Abstract
Natural language processing (NLP) tasks have achieved excellent performance in many fields, including semantic understanding, automatic summarization, and image recognition. However, most neural network models for NLP extract text features in a fine-grained way, which can hinder the ability to grasp the meaning of the text from a global perspective. To alleviate this problem, this paper proposes a Multi Model Nonlinear Fusion (MMNF) model that combines traditional statistical methods with deep learning. The model utilizes the Jaccard coefficient based on part of speech (POS), Term Frequency-Inverse Document Frequency (TF-IDF), and a word2vec-CNN algorithm to measure sentence similarity. By inputting weighted vectors into a fully connected neural network, the system achieves a more holistic understanding. Experimental results show that the MMNF model achieves 84% accuracy and an F1 value of 75%.
Introduction
Determining semantic similarity between sentences is a cornerstone of Natural Language Processing (NLP). While deep learning has significantly advanced the field, many modern models focus predominantly on "fine-grained" local details—analyzing text word-by-word or character-by-character. This approach often misses the "coarse-grained" global context that traditional statistical methods are well-suited to capture. This paper introduces a hybrid framework designed to bridge this gap by fusing statistical insights with deep semantic features.
The Multi Model Nonlinear Fusion (MMNF) Framework
The proposed MMNF model calculates similarity scores using three distinct sub-models, each offering a different perspective on the text:
- TF-IDF (Term Frequency-Inverse Document Frequency): This classic statistical measure evaluates the importance of words within a sentence pair based on their frequency across a corpus. It provides a global view of keyword relevance.
- Improved Jaccard Coefficient: Standard Jaccard similarity measures word overlap. The authors enhance this by incorporating Part-of-Speech (POS) weighting. By assigning higher weights to subjects and predicates compared to adjectives or adverbs, the model prioritizes the grammatical core of the sentence.
- word2vec-CNN with Attention: This component handles "fine-grained" semantic extraction. It uses word2vec embeddings and a Convolutional Neural Network (CNN) integrated with an attention mechanism. This setup considers both the semantic meaning of words and their relative positions (using edit distance) to build a robust feature matrix.
The final classification is performed by a Fully Connected Neural Network (FCNN). Instead of simple averaging, the FCNN learns the optimal nonlinear weights for combining the scores from the three sub-models, allowing the system to adapt to complex linguistic patterns.
Experimental Results
The MMNF model was evaluated on two major benchmarks:
- Ant Financial NLP Dataset: A large-scale Chinese dataset with 110,000 sentence pairs from customer service interactions.
- STS (Semantic Textual Similarity) Datasets (2012–2015): Standard English benchmarks used for comparison against state-of-the-art models like BERT.
On the Ant Financial dataset, the MMNF model achieved an accuracy of 84% and an F1-value of 75%. Notably, the fusion model outperformed its individual components; for instance, the word2vec-CNN alone reached 80% accuracy, demonstrating the value of the statistical "coarse-grained" features. On the STS benchmarks, MMNF outperformed basic BERT implementations (BERT[CLS] and Average BERT) in several instances, though it trailed specialized models like Sentence-BERT (SBERT) in later years.
Key Insights
- Global vs. Local Features: Statistical methods like TF-IDF provide a necessary global context that deep learning models sometimes overlook.
- Grammatical Importance: Weighting word overlaps based on their grammatical roles (POS) significantly improves the model's ability to distinguish meaning.
- Position Sensitivity: Incorporating word location information within the CNN component enhances feature extraction by accounting for how word order affects semantics.
Conclusion and Future Work
The MMNF model successfully demonstrates that combining traditional statistics with deep learning leads to a more comprehensive understanding of sentence similarity. A current limitation is the use of static word embeddings (word2vec), which cannot handle polysemy (words with multiple meanings). Future research will explore integrating dynamic embedding models like ELMo or BERT into the fusion framework to further enhance contextual semantic capture while retaining the benefits of global statistical features.
Summary updated