Statistical Relational Learning with Formal Ontologies

Achim Rettinger, Matthias Nickles, Volker Tresp

Abstract

In many domains, prior knowledge exists as "hard" logical constraints that define valid relationships between entities. Traditional Statistical Relational Learning (SRL) often treats such knowledge as soft priors, potentially leading to logically inconsistent predictions. This paper introduces the Infinite Hidden Semantic Model (IHSM), which embeds formal ontologies directly into the learning process. By integrating a Description Logic reasoner with a Dirichlet Process-based clustering model, the IHSM ensures that all predicted relations are logically valid while improving predictive performance and the interpretability of latent structures.

Introduction

Machine learning typically treats prior knowledge as a flexible guide. However, in fields like medicine or the Semantic Web, rigid logical rules—formal ontologies—govern the domain. Standard SRL models excel at identifying patterns in networked data but often fail to respect these hard constraints. For instance, a model might predict a child attends a university based on proximity, ignoring ontological rules that forbid such an association. The challenge lies in bridging the gap between flexible probabilistic learning and uncompromising formal logic.

Methodology: The Infinite Hidden Semantic Model

The IHSM builds upon the Infinite Hidden Relational Model (IHRM), where entities are assigned to latent clusters, and relationship probabilities depend on cluster memberships. The IHSM enhances this by adding a "Semantic" layer using $SHOIN(D)$ Description Logic.

The domain ontology schema used in the study, defining entities like Person, School, and Location and their valid relationships.

Model Architecture and Constraint Enforcement

The core of the IHSM is a generative model where latent variables $Z$ represent cluster assignments for entities. These assignments determine the likelihood of observed attributes $A$ and relations $R$.

Plate notation for the Infinite Hidden Semantic Model (IHSM), illustrating the dependencies between latent clusters (Z), attributes (A), and relations (R).

During the learning process, specifically within the Gibbs sampling loop, the model consults a formal reasoner (e.g., Pellet). When considering a cluster assignment or relationship prediction, the reasoner checks for consistency against the ontology. If a choice violates a logical rule, that path is blocked. This tight integration ensures that constraints influence the latent cluster assignments themselves, forcing the model to discover a hidden structure that is both statistically probable and logically sound.

Experimental Evaluation

The authors evaluated the IHSM using the LJ-FOAF dataset, a real-world social network derived from LiveJournal profiles using the "Friend-of-a-Friend" ontology. The dataset comprised 638 persons, 747 schools, and 200 locations. To rigorously test the model, the researchers added custom "hard" constraints, such as age-based restrictions on school attendance.

Results and Key Insights

The IHSM demonstrated significant advantages over the standard IHRM:

  • Improved Predictive Accuracy: For the attends relation, the Area Under the ROC Curve (AUC) rose from 0.577 to 0.608. Interestingly, accuracy also improved for unconstrained relations like knows (0.813 to 0.824), indicating that constraints propagate benefits across the entire network.
  • Guaranteed Consistency: The IHSM produced zero logically inconsistent predictions, whereas the standard model frequently violated ontological rules.
  • Enhanced Interpretability: Hard constraints act as guardrails, zeroing out impossible relations and focusing probabilistic energy on valid configurations. This often forces the model to create more meaningful latent clusters (e.g., separating "children" from "adults" to satisfy school attendance rules).

Conclusion

The IHSM successfully integrates deductive reasoning into the MCMC loop of a relational learning model. While this integration introduces computational overhead—making training approximately six times slower—it guarantees logical consistency and improves overall model performance. Future work involves "knowledge extraction," where learned latent structures are used to discover new formal rules, creating a feedback loop between symbolic logic and statistical learning.

Summary updated

Abstract

We propose a learning approach for integrating formal knowledge into statistical inference by exploiting ontologies as a semantically rich and fully formal representation of prior knowledge. The logical constraints deduced from ontologies can be utilized to enhance and control the learning task by enforcing description logic satisfiability in a latent multi-relational graphical model. To demonstrate the feasibility of our approach we provide experiments using real world social network data in form of a $\mathcal{SHOIN}(D)$ ontology. The results illustrate two main practical advancements: First, entities and entity relationships can be analyzed via the latent model structure. Second, enforcing the ontological constraints guarantees that the learned model does not predict inconsistent relations. In our experiments, this leads to an improved predictive performance.
BibTeX
@inproceedings{rettinger2009statistical,
  author = {Achim Rettinger and Matthias Nickles and Volker Tresp},
  title = {{Statistical Relational Learning with Formal Ontologies}},
  booktitle = {ECML/PKDD},
  year = {2009},
  doi = {10.1007/978-3-642-04174-7\_19},
}
Figure