A Multiple-Domain Ontology Builder

Sara Salem, Samir AbdelRahmanCOLING 2010

TL;DR

This paper tackles the problem of building a single, unified ontology from a corpus containing documents from multiple, distinct domains. They propose a novel two-layer ontology builder that first extracts separate ontologies for each domain and then integrates them with a general layer, leading to more accurate concept representation and improved question answering.

What Problem Does This Paper Address?

Building ontologies, which are structured representations of knowledge, is crucial for applications like the Semantic Web, information retrieval, and question answering. However, existing automatic ontology builders typically assume a single-domain corpus. When applied to a multi-domain corpus, they create a single, large ontology. This approach leads to two major issues: 1) Concept Imprecision: Common terms that have different meanings across domains (e.g., "wall street" as a financial institution vs. a movie) are forced into a single, often inaccurate, definition. 2) Misleading Concept Significance: The relevance weights assigned to concepts don't accurately reflect their importance within specific domains, potentially burying crucial domain-specific terms. This paper aims to solve these misconceptions by creating domain-specific ontologies that better capture nuanced meanings.

Methods

The authors propose a two-layer ontology builder.

  • Conceptual Layer: This layer focuses on extracting domain-specific concepts and relations.
  • Clustering: The first step is to cluster documents from the multi-domain corpus into domain-specific groups. They use an agglomerative hierarchical clustering technique, which doesn't require knowing the number of domains beforehand. Document similarity is calculated using TF-IDF weights and Cosine Similarity.
  • Ontology Extraction: For each cluster (domain), they leverage an existing ontology builder, Text2Onto, to extract a separate domain-specific ontology.
  • General Layer: This layer captures general domain information, such as named entities (persons, locations, organizations, dates) and their relations.
  • Mixed Tagging: To overcome Text2Onto's limitations in extracting general domain information, they introduce a "Mixed Tagging" approach. This uses standard POS tags along with custom tags like CONCEPT, PERSON, LOCATION, ORGANIZATION, and DATE.
  • Pattern-Based Relation Extractor: A flexible relation extractor uses regular expressions (patterns) over these mixed tags to identify relationships between concepts and named entities, including those not well-handled by Text2Onto.

The overall system integrates these layers to create a more comprehensive and accurate representation of knowledge from multi-domain corpora.

Datasets

The primary dataset used for evaluation is the AQUAINT corpus. This is an English news corpus comprising documents from the New York Times News Service, Xinhua News Service, and Associated Press Worldstream News Service. For question answering evaluation, they utilize a subset of the AQUAINT corpus and questions from the TREC 2004 Question Answering track. They also mention using a subset of DMOZ data for clustering experiments.

Key Insights and Ideas

  • The "one ontology fits all" fallacy: The paper strongly argues that forcing multi-domain data into a single ontology leads to significant semantic inaccuracies and misrepresentations of concept importance.
  • Domain-specific ontologies are key: By first creating separate ontologies for each domain, the system can preserve the unique semantics of concepts within their respective contexts.
  • General information matters: Simply extracting core domain concepts isn't enough. Capturing general entities and their relationships (like who met where and when) is crucial for richer knowledge representation and better question answering.
  • Flexible pattern matching for relations: The proposed "Mixed Tagging" and pattern-based relation extractor are vital for overcoming the limitations of existing tools in capturing diverse real-world relationships, especially those involving named entities.

Key Results

  • Improved Concept Weighting: Their builder assigns higher weights to domain-specific concepts within their respective ontologies compared to Text2Onto's single ontology. This provides a more accurate reflection of concept importance.
  • Accurate Modeling of Common Concepts: For concepts with different meanings across domains (e.g., "Wall Street"), their builder successfully maintains these distinct definitions, whereas Text2Onto collapses them into a single definition.
  • Enhanced Question Answering:
    • Definition Questions: Achieved an F-Measure of 1.0, significantly outperforming Text2Onto's 0.5, due to preserving multiple definitions.
  • Single-Domain Questions: Achieved an accuracy of 68% compared to Text2Onto's 0.05%. This highlights the benefit of the general layer for accurately answering "Who, When, Where" questions.
  • Cross-Domain Questions: Achieved an F-Measure of 0.44, while Text2Onto answered none, again demonstrating the value of the general layer and domain-specific ontologies.
  • Information Retrieval Performance: Initial results using 220 queries on 600 AQUAINT documents show an F-Measure of 0.35 against Lucene's 0.18, suggesting potential for improving IR systems.

Limitations and Future Work

  • Error Sources in QA: The authors identify several sources of errors in their question answering system:
  • Pattern Matching: 65% of errors occur because relations don't match predefined patterns (elements too far apart).
  • NER Errors: 16% are due to missed or wrongly tagged Named Entities.
  • Co-reference: 12% of errors stem from unhandled co-references.
  • Low Answer Weight: 7% of answers are extracted with lower weights than the correct answer.
  • Future Work:
    • Automatic Pattern Learning: Improve the relation extractor by automatically learning patterns from text.
  • Co-reference Resolution: Implement co-reference handling to improve accuracy.
  • Information Retrieval Enhancement: Further enhance their Information Retrieval system.

Summary updated

BibTeX
@inproceedings{salem2010multiple,
  author = {Sara Salem and Samir AbdelRahman},
  title = {{A Multiple-Domain Ontology Builder}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2010},
}
Figure