Math-KG: Construction and Applications of Mathematical Knowledge Graph

Jianing WangarXiv 2022 · OpenReview

TL;DR

This paper introduces Math-KG, an automatically constructed mathematical knowledge graph designed to combat information overload in online math education. By integrating vast amounts of unstructured data using NLP techniques, Math-KG enables applications like personalized fault analysis and semantic search for mathematical concepts.

What Problem Does This Paper Address?

The rapid growth of online education platforms has led to an explosion of readily accessible math resources. However, these platforms often fail to integrate the massive amounts of unstructured information available, resulting in "information overload" and "knowledge trek" for students. This means students are presented with a deluge of disconnected facts and concepts, making it difficult to navigate and truly understand the subject. Existing knowledge graphs have been successful in fields like medicine and finance, but mathematical education has remained a largely untapped area for this technology.

Methods

The authors propose a pipeline method for constructing Math-KG, which involves two main stages: Corpus Preprocessing and Information Extraction.

  • Corpus Preprocessing: This stage focuses on acquiring and cleaning mathematical data. They crawl semi-structured and unstructured data from sources like Baidu Baike and Wikipedia. For each entity, they extract categories, abstracts, main bodies, infoboxes, and indexes. Text is then processed using NLP tools (like Spacy) for sentence splitting, tokenization, POS tagging, and dependency parsing. Mathematical formulas are converted to LaTeX using MathOCR.
  • Information Extraction: This stage aims to extract structured triples and build the knowledge graph. It comprises three sub-tasks:
  • Knowledge Entity Recognition (KER): They define two entity types: "Conceptual Entity" (e.g., triangle) and "Theorem Entity" (e.g., Pythagorean theorem). They use a distant supervision strategy with BERT for fine-tuning to identify these entities.
  • Entity Relation Classification (ERC): Six relation types are defined (dependencies, affiliation, equivalence, antisense, synonyms, has properties), leading to 12 directed relation types. They leverage explicit triples from seed data and then use a self-supervision method and BERT with a sentence-level attention layer for relation classification.
  • Knowledge Fusion: To enhance accuracy, they employ entity linking tools (TagMe) and reference resolution techniques (Neuralcoref) to align with existing knowledge bases and ensure consistency.

Datasets

The paper utilizes data from:

  • Baidu Baike: A Chinese online encyclopedia.
  • Wikipedia: A collaborative, multilingual online encyclopedia.

They also introduce two core datasets for training and evaluation:

  • Knowledge Entity Recognition (KER) dataset: Contains 8,950 samples.
  • Entity Relation Classification (ERC) dataset: Contains 13,365 samples.

These datasets, along with corresponding codes, are publicly available on GitHub.

Key Insights and Ideas

  • Bridging the Gap in Math Education: The core insight is that knowledge graphs, proven effective in other domains, can significantly improve the structure and accessibility of mathematical knowledge, addressing the persistent problem of information overload in online learning.
  • Dual Entity Types for Math: Defining distinct "Conceptual Entities" and "Theorem Entities" is crucial for capturing the specific nature of mathematical knowledge, differentiating between fundamental concepts and the rules or methods that operate on them.
  • Pipeline for Scalability: The proposed pipeline approach, from data acquisition to knowledge fusion, offers a scalable and automated method for building a comprehensive mathematical knowledge graph, reducing manual effort.
  • Application-Driven Validation: Demonstrating the utility of Math-KG through practical applications like "Faults Analysis" and "Semantic Search" provides strong evidence of its value beyond just graph construction.

Key Results

  • Quantitative Performance: The F1 scores for Knowledge Entity Recognition (KER) and Entity Relation Classification (ERC) are reported as 92.62% and 85.09%, respectively. While acknowledging room for improvement, these scores indicate a solid foundation for the constructed graph.
  • Graph Size: The constructed Math-KG contains 1905 mathematical knowledge point entities and 8,019 triples. This includes 1337 concept entities and 568 theorem entities.
  • Relation Distribution: The most frequent relation types are dependencies (2016 triples) and affiliation (2609 triples), highlighting the interconnectedness of mathematical concepts.
  • Application Demonstrations:
    • Faults Analysis: The system can identify knowledge points a student struggles with by analyzing their historical performance and suggesting related concepts that might be the root cause of errors.
  • Semantic Search: The system can answer natural language queries by recognizing topic entities and retrieving relevant multi-hop sub-graphs from Math-KG, exemplified by finding the relationship between "triangle" and "circumscribed circle radius."

Limitations and Future Work

  • Accuracy Improvement: The authors acknowledge that the current algorithm has room for improvement in terms of accuracy for both KER and ERC.
  • Future Work: The authors plan to:
  • Improve the accuracy and efficiency of information extraction.
  • Apply Math-KG to other disciplines, such as physics and chemistry.

Summary updated

Abstract

Recently, the explosion of online education platforms makes a success in encouraging us to easily access online education resources. However, most of them ignore the integration of massive unstructured information, which inevitably brings the problem of \textit{information overload} and \textit{knowledge trek}. In this paper, we proposed a mathematical knowledge graph named Math-KG, which automatically constructed by the pipeline method with the natural language processing technology to integrate the resources of the mathematics. It is built from the corpora of Baidu Baike, Wikipedia. We implement a simple application system to validate the proposed Math-KG can make contributions on a series of scenes, including faults analysis and semantic search. The system is publicly available at GitHub \footnote{\url{https://github.com/wjn1996/Mathematical-Knowledge-Entity-Recognition}.}.
BibTeX
@misc{wang2022math,
  author = {Jianing Wang},
  title = {{Math-KG: Construction and Applications of Mathematical Knowledge Graph}},
  year = {2022},
  eprint = {2205.03772},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.03772},
}
Figure