Abstract
Knowledge extraction from Chinese scientific and technological texts is hindered by linguistic nuances and complex technical jargon. Traditional pipeline methods often suffer from error propagation and lose correlations between entity recognition and relation extraction. This paper proposes MBGAB, a joint extraction model that integrates character and word-level embeddings ("Word Mixing") with a Bi-GRU encoder and self-attention mechanism. A key innovation is the introduction of a biased loss function to prioritize meaningful entities over background text. Experimental results on a Chinese technical corpus demonstrate that MBGAB significantly outperforms baseline models, achieving an F1 score of 56.1%.
Introduction
Knowledge extraction—the process of transforming unstructured text into structured (entity-relation-entity) triples—is fundamental for building domain-specific knowledge graphs. In the context of Chinese scientific literature, two primary challenges exist. First, the "pipeline problem" in traditional systems leads to slow processing and a lack of feedback between entity and relation identification. Second, the absence of spaces in Chinese text causes "boundary segmentation errors," where technical terms are incorrectly split, leading to semantic loss.
Methodology: The MBGAB Model
The proposed MBGAB (Word Mixing, Bi-GRU, Attention, and Bias weight) model addresses these challenges through a multi-layered architecture designed for joint extraction.
Word Mixing Vector Mapping
To mitigate segmentation errors, the model employs a hybrid embedding approach. It combines character-level vectors with word-level vectors derived from pre-trained Word2Vec models. This ensures that even if a word boundary is ambiguous, the character-level information preserves the underlying semantic meaning.
Bi-GRU Encoder and Self-Attention
A bidirectional Gated Recurrent Unit (Bi-GRU) processes the text to capture bidirectional context. To handle long-distance dependencies common in complex technical sentences, a self-attention layer is integrated. This layer allows the model to relate distant words regardless of their position in the sequence.
GRU Decoder with Bias Weight
The decoder uses a GRU-based sequence labeling approach. A critical component is the introduction of a bias weight ($\alpha$) in the loss function. Since most tokens in a sentence are "Outside" (O) tags, models often become biased toward predicting "O." The $\alpha$ parameter forces the model to pay more attention to the rarer but more significant entity and relation tags.
Experimental Results and Discussion
The model was evaluated using a Chinese corpus from the Baidu Language and Intelligent Technology Competition, containing over 24,000 training sets and 19 relationship types.
Impact of Bias Weight ($\alpha$)
The study found that the bias weight is crucial for balancing precision and recall. As shown in the performance analysis, the model's F1 score and recall improve as $\alpha$ increases from 1.
The "sweet spot" was identified at $\alpha = 3$, where the model achieves the best balance. Beyond this point, precision begins to decline as the model may over-predict relations in its attempt to avoid "O" tags.
Comparative Performance
MBGAB achieved an F1 score of 56.1% and an accuracy of 65.3%, outperforming the pipeline-based FCM model (41.3% F1) and standard Bi-GRU joint models (51.3% F1). Qualitative analysis showed the model's high proficiency in identifying professional vocabulary and complex relationships in technical case studies.
Conclusion
The MBGAB model demonstrates that joint extraction combined with hybrid embeddings and biased loss functions is highly effective for technical Chinese text. Future work will focus on scaling the model to handle thousands of niche relations and addressing overlapping relations where a single entity participates in multiple triples.
Summary updated