Abstract
Concept graphs are created as universal taxonomies for text understanding in the open-domain knowledge. The nodes in concept graphs include both entities and concepts. The edges are from entities to concepts, showing that an entity is an instance of a concept. In this paper, we propose the task of learning interpretable relationships from open-domain facts to enrich and refine concept graphs. The Bayesian network structures are learned from open-domain facts as the interpretable relationships between relations of facts and concepts of entities. We conduct extensive experiments on public English and Chinese datasets. Compared to the state-of-the-art methods, the learned network structures help improving the identification of concepts for entities based on the relations of entities on both datasets.
The Sparsity Challenge in Concept Graphs
Building a comprehensive "Concept Graph"—a map where nodes represent entities and concepts, and edges denote "is-a" relationships—is fundamental for advanced natural language understanding. Traditionally, these graphs are constructed using pattern-based extraction (e.g., "Apples, such as Fuji" implies Fuji is-a Apple).
However, a significant "sparsity" problem exists. Analysis of millions of sentences from Wikipedia and Baidu Baike reveals that an entity and its corresponding concept co-occur in the same sentence less than 11% of the time. For instance, a text might state "Anderson hosted Whose Line is it Anyway?" without ever explicitly mentioning "Anderson is a presenter." Traditional methods fail to bridge this gap because the explicit "is-a" link is absent. The challenge lies in leveraging the rich relational information found in Open Information Extraction (OIE) facts—triples like (subject, predicate, object)—to infer these missing links.
Methodology: Bayesian Network Structure Learning
To address this, we propose a workflow that utilizes Bayesian Network Structure Learning (BNSL) to discover interpretable relationships between relations and concepts.
The process involves four key stages:
- Fact Extraction: Raw text is converted into triples $(s, r, o)$ using OIE tools.
- View Splitting: Relations are categorized into a Subject-Relation View (actions performed by the entity) and an Object-Relation View (actions performed upon the entity).
- Structure Learning: Since learning a full Bayesian Network across millions of variables is NP-hard, we decompose the problem. For each concept, we select the top $K$ most relevant relations and apply the Chow-Liu tree algorithm. This algorithm constructs a "maximum spanning tree" of dependencies, creating a human-readable map of how specific actions (e.g., "hosting") correlate with specific concepts (e.g., "presenter").
- Inference: The learned network allows us to calculate the probability of an entity belonging to a concept based on its observed relations.
Experimental Insights and Results
The model was evaluated on large-scale English and Chinese datasets, including 15 million OIE facts and the Microsoft Concept Graph.
Key Findings:
- Relations as Proxies: An entity's behavior is a powerful signal for its type. Winning a "British Comedy Award" is a strong indicator of being a "comedian," even without explicit textual confirmation.
- Subject vs. Object Informativeness: Subject-view relations were found to be generally more informative than Object-view relations, likely because authors tend to provide more defining predicates for sentence subjects.
- Interpretability: Unlike "black-box" neural models, the Bayesian structures are human-readable. The resulting trees clearly show which relations (e.g., "host," "winner") justify a classification.
- Performance and Robustness: BNSL consistently outperformed SVM and RNN baselines. Notably, it maintained high accuracy even with extremely sparse input data. When ensembled with path-based models like HypeNet, the combined approach achieved an F1-score of 92.43% on English datasets, demonstrating that global relational facts and local sentence patterns provide complementary knowledge.
Conclusion
By applying Bayesian Structure Learning to open-domain facts, we can effectively bridge the gap in concept graphs caused by co-occurrence sparsity. The resulting models are not only high-performing but also provide interpretable justifications for their classifications. Future work will explore more complex Directed Acyclic Graph (DAG) structures and methods to handle even noisier open-domain data.
Summary updated