TL;DR
This paper introduces a novel semi-automatic method for assessing the quality of Linked Data that doesn't require a pre-existing ontology. By analyzing data patterns directly, their approach can evaluate data quality even in resources like Korean DBpedia, which lack ontologies, achieving high accuracy and consistency.
What Problem Does This Paper Address?
The explosion of Linked Data on the Semantic Web has brought immense benefits, but it has also highlighted a critical challenge: data quality. Many existing methods for assessing this quality rely heavily on having a structured data schema, such as an ontology. However, a significant portion of Linked Data, especially in non-English languages or emerging domains, lacks these ontologies. This creates a gap where robust quality assessment is difficult or impossible, hindering the trustworthiness and usability of these valuable data resources. The authors aim to bridge this gap by developing a method that can assess data quality directly from the data itself, without needing an ontology.
Methods
The proposed approach is a three-step semi-automatic process:
- Pattern Analysis: The system first analyzes the structure of the Linked Data to identify common patterns within properties. This involves examining how subjects are linked to objects via predicates.
- Test Case Pattern Generation: Based on the observed patterns, the algorithm automatically generates "test case patterns." The core assumption here is that frequently occurring data patterns are more likely to be correct. The algorithm identifies patterns that appear more frequently than a dynamically determined threshold, effectively creating a set of rules that represent "valid" data. This step is crucial as it bypasses the need for manual rule creation or ontology-based definitions.
- Data Quality Evaluation: The generated test case patterns are then applied to the Linked Data to evaluate its quality. The evaluation focuses on dimensions like domain (correct type of subject for a property) and range (correct type of object for a property), and datatype.
The novelty lies in the automatic generation of these test case patterns directly from the data, making it adaptable to resources without ontologies.
Datasets
- English DBpedia: Used for validating the accuracy of the generated test case patterns by comparing them against patterns derived from its existing ontology.
- Korean DBpedia: Used for evaluating the quality assessment accuracy of the proposed method in a real-world scenario where no ontology is available.
Key Insights and Ideas
- Ontology-Free Quality Assessment: The most significant insight is that high-quality data assessment is achievable even without a formal ontology, by leveraging the inherent patterns within the data itself.
- Frequency as a Proxy for Validity: The assumption that frequently occurring data patterns are more credible is a powerful heuristic. This allows the system to infer quality rules from the data's structure.
- Adaptability to Non-English Data: The method's success with Korean DBpedia demonstrates its potential for global applicability, addressing a key limitation of many existing ontology-dependent tools.
- Pattern Equivalence vs. Literal Equivalence: The paper highlights a challenge where patterns might be semantically equivalent but expressed differently (e.g.,
xsd:Stringvs.rdf:langString), suggesting future work on mapping mechanisms.
Key Results
- High Pattern Generation Rate: The approach achieved high pattern generation rates (up to 99%) when comparing generated patterns with existing ones in English DBpedia.
- Good Consistency: Consistency between the generated patterns and ontology-derived patterns was substantial, reaching up to 89.35% for Domain Quality Patterns (DQP).
- Accurate Quality Assessment: For Korean DBpedia, the method achieved a high average F1-measure of approximately 0.7 (up to 0.79), indicating reliable quality assessment even without an ontology.
- Identification of Quality Issues: The evaluation on Korean DBpedia successfully identified various quality problems, including range violations and issues related to missing
rdf:typedefinitions, demonstrating the practical utility of the approach.
Limitations and Future Work
The authors acknowledge several limitations and suggest future directions:
- Manual Pattern Definition: While test case patterns are generated automatically, the initial definition of pattern structures (like DQP, RQP, TQP) still requires domain expert input.
- Mapping Equivalent Data Types: The paper notes that patterns with semantically equivalent but literally different data types (e.g.,
xsd:Stringvs.rdf:langString) can lead to lower consistency scores. Future work could involve developing a mechanism to map these equivalent types. - External Resource Utilization: To further address schema-related quality problems, the authors plan to explore the use of external resources like WordNet and Thesaurus.
- Broader Pattern Set: Investigating and incorporating a wider variety of patterns is suggested to enhance the scope of quality assessment.
- Complete Validation System: The ultimate goal is to develop a comprehensive system for validating the trustworthiness of triples.
Summary updated