Asking Effective and Diverse Questions: A Machine Reading Comprehension based Framework for Joint Entity-Relation Extraction

Tianyang Zhao, Zhao Yan, Yunbo Cao, Zhoujun Li

Abstract

Entity-Relation Extraction (ERE) is a fundamental task in information extraction. This paper presents a novel joint extraction framework that reformulates ERE as a multi-turn Machine Reading Comprehension (MRC) task. By introducing diverse question generation and a relation prediction filtering mechanism, the proposed model, MRC4ERE++, addresses the limitations of semantic rigidity and computational inefficiency found in previous MRC-based approaches. Experimental results on ACE05 and CoNLL04 benchmarks demonstrate that the framework achieves state-of-the-art performance by capturing deeper semantic nuances and filtering out computational noise.

Introduction

Entity-Relation Extraction (ERE) involves identifying entities (e.g., "U.S. soldiers") and their semantic relationships (e.g., "employed by") within unstructured text. Traditional pipeline methods often suffer from error propagation between the entity recognition and relation extraction stages. While joint models and earlier Machine Reading Comprehension (MRC) frameworks have been proposed to mitigate this, they often rely on static, single-query templates (e.g., "Find people") that fail to capture the linguistic variety of natural language. Furthermore, exhaustively querying every possible relation for every entity pair introduces significant computational noise and inefficiency.

Methodology

The MRC4ERE++ framework utilizes a BERT-based architecture to perform joint extraction through a series of structured steps designed to prune the search space and enhance semantic capture:

  1. Diverse Question Answering (DQA): Instead of a single rigid query, the model generates a group of diverse questions for each entity type (e.g., "Who is mentioned?", "Find people...", "Which words are person entities?"). This multi-perspective approach ensures higher robustness.
  2. Answer Ensemble Strategy: To reconcile potentially differing outputs from diverse questions, a weighted voting scheme is employed. Weights are determined by the historical performance (F1 score) of specific question templates during training.
  3. Relation Prediction (RP): Before searching for a "tail" entity, the model performs a quick classification to predict which relations are actually plausible for a given "head" entity. This filters out irrelevant relations, significantly reducing negative samples.
  4. Joint Training: All components—head entity extraction, relation prediction, and tail entity extraction—are optimized together in a single loss function, allowing the model to learn the interactions between these sub-tasks.
The BERT-based architecture of MRC4ERE++, illustrating the flow from diverse questions to relation prediction and entity extraction.

Experimental Evaluation

The model was evaluated on two widely recognized benchmarks:

  • ACE05: A complex dataset containing 7 entity types and 6 relation types.
  • CoNLL04: A dataset featuring 4 entity types and 5 relation types.

Key insights from the experiments revealed that natural language templates (human-like questions) significantly outperformed "pseudo-questions" (e.g., "entity:person"). This suggests that the pre-trained knowledge in models like BERT is best activated by grammatically correct, semantically rich queries.

Results and Discussion

MRC4ERE++ achieved significant improvements over existing state-of-the-art models. On the ACE05 dataset, it reached a relation F1 score of 62.1%, a +1.9% gain over the previous best MRC-based baseline. On CoNLL04, the improvement was even more pronounced, with a +3.0% increase to 71.9% F1.

Comparison of MRC4ERE and MRC4ERE++ against baseline models on the ACE05 and CoNLL04 datasets.

Ablation studies confirmed that the "Diverse Question" component is critical; removing it led to a measurable performance drop, validating the hypothesis that multiple query perspectives are more robust than any single "best" question.

Conclusion

The proposed framework demonstrates that treating entity-relation extraction as a conversation with diverse queries can effectively capture complex semantic nuances while pruning irrelevant search spaces. While the current reliance on human-defined templates and the computational cost of multiple BERT passes are areas for future optimization, the success of MRC4ERE++ sets a new standard for joint extraction tasks. Future directions include automated question generation and extending the framework to handle cross-sentence relation extraction.

Summary updated

Abstract

Recent advances cast the entity-relation extraction to a multi-turn question answering (QA) task and provide an effective solution based on the machine reading comprehension (MRC) models. However, they use a single question to characterize the meaning of entities and relations, which is intuitively not enough because of the variety of context semantics. Meanwhile, existing models enumerate all relation types to generate questions, which is inefficient and easily leads to confusing questions. In this paper, we improve the existing MRC-based entity-relation extraction model through diverse question answering. First, a diversity question answering mechanism is introduced to detect entity spans and two answering selection strategies are designed to integrate different answers. Then, we propose to predict a subset of potential relations and filter out irrelevant ones to generate questions effectively. Finally, entity and relation extractions are integrated in an end-to-end way and optimized through joint learning. Experiment results show that the proposed method significantly outperforms baseline models, which improves the relation F1 to 62.1% (+1.9%) on ACE05 and 71.9% (+3.0%) on CoNLL04. Our implementation is available at https://github.com/TanyaZhao/MRC4ERE.
BibTeX
@inproceedings{zhao2020asking,
  author = {Tianyang Zhao and Zhao Yan and Yunbo Cao and Zhoujun Li},
  title = {{Asking Effective and Diverse Questions: A Machine Reading Comprehension based Framework for Joint Entity-Relation Extraction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/546},
}
Figure