OntoLAMA: LAnguage Model Analysis for Ontology Subsumption Inference

Yuan He, Jiaoyan Chen, Ernesto Jiménez-Ruiz, Hang Dong, Ian HorrocksZenodo 2023 · DOI · OpenAlex

Abstract

About OntoLAMA is a set of language model (LM) probing datasets for ontology subsumption inference. The work follows the "LMs-as-KBs" literature but focuses on conceptualised knowledge extracted from formalised KBs such as the OWL ontologies. Specifically, the subsumption inference (SI) task is introduced and formulated in the Natural Language Inference (NLI) style, where the sub-concept and the super-concept involved in a subsumption axiom are verbalised and fitted into a template to form the premise and hypothesis, respectively. The sampled axioms are verified through ontology reasoning. The SI task is further divided into Atomic SI and Complex SI where the former involves only atomic named concepts and the latter involves both atomic and complex concepts. Real-world ontologies of different scales and domains are used for constructing OntoLAMA and in total there are four Atomic SI datasets and two Complex SI datasets. Dataset Source #Concepts #EquivAxioms #Datasets(Train/Dev/Test) Schema.org 894 N/A Atomic SI: 808/404/2, 830 DOID 11,157 N/A Atomic SI: 90,500/11,312/11,314 FoodOn 30,995 2,383 Atomic SI: 768,486/96,060/96,062 Complex SI: 3,754/1,850/13,080 GO 43,303 11,456 Atomic SI: 772,870/96,608/96,610 Complex SI: 72,318/9,040/9,040 MNLI N/A N/A biMNLI: 235,622/26,180/12,906 Citation The relevant paper has been accepted at Findings of ACL 2023: https://aclanthology.org/2023.findings-acl.213/. ```@inproceedings{he2023language, title={Language Model Analysis for Ontology Subsumption Inference}, author={He, Yuan and Chen, Jiaoyan and Jimenez-Ruiz, Ernesto and Dong, Hang and Horrocks, Ian}, booktitle={Findings of the Association for Computational Linguistics: ACL 2023}, pages={3439--3453}, year={2023} }``` Links See instructions at: https://krr-oxford.github.io/DeepOnto/ontolama/ We have made available a convenient access of these datasets through Huggingface: https://huggingface.co/datasets/krr-oxford/OntoLAMA The arxiv version is available at: https://arxiv.org/abs/2302.06761 Contact Yuan He (yuan.he(at)cs.ox.ac.uk)
BibTeX
@inproceedings{he2023ontolama,
  author = {Yuan He and Jiaoyan Chen and Ernesto Jiménez-Ruiz and Hang Dong and Ian Horrocks},
  title = {{OntoLAMA: LAnguage Model Analysis for Ontology Subsumption Inference}},
  booktitle = {Zenodo},
  year = {2023},
  doi = {10.5281/zenodo.6480540},
}

Loading related content...

Figure