TL;DR
This paper introduces a novel "Dominance Metric" to quantify the importance of concepts within semantic taxonomies, addressing the challenge of fragmented and proprietary classification systems in scholarly publishing. By applying this metric to large-scale taxonomies like Microsoft Academic Graph Fields of Study and MeSH, the authors demonstrate how to create more compact, refined, and universally applicable taxonomic structures.
What Problem Does This Paper Address?
In the realm of scholarly publishing, semantic taxonomies are crucial for organizing and discovering content, as well as for powering recommendation systems. However, many publishers rely on proprietary taxonomies, leading to fragmented "data silos" that hinder cross-publisher discoverability and analysis. Existing publicly available taxonomies, while comprehensive, may not always be sufficiently tailored or may be too unwieldy for practical use. The creation and maintenance of these taxonomies are also labor-intensive and costly. This paper tackles the problem of how to systematically identify and rank the most important concepts within these large, often proprietary, taxonomic structures to create more manageable and effective classification systems.
Methods
The core contribution is the proposed "Dominance Metric" methodology. This metric is designed to be modular and scalable, applicable to any hierarchical taxonomy. It combines several factors to assess concept importance:
- Depth: The overall depth of the taxonomy.
- Level: The specific level of a concept within the hierarchy.
- Descendants (direct and inferred): The total number of child concepts, including those nested deeper.
- Tagged Objects (direct and inferred): The number of content items associated with a concept and its descendants.
The metric aims to go beyond simple popularity (number of tagged objects) by incorporating structural and topological properties. The authors also introduce domain-specific factors for their case studies:
- MAG FoS Taxonomy: "UMLS relation" (whether a concept is linked to the Unified Medical Language System) and "Source relation" (whether linked to external knowledge sources like Wikipedia).
- MeSH Taxonomy: "Registry relation" (whether a concept is linked to external registries like CAS or NCBI).
The paper also details a "cleansing process" involving deduplication of concepts (using Levenshtein distance) and refinement of hierarchical relations (using Cosine Similarity on word embeddings) to improve the quality of the derived taxonomies.
Datasets
- Microsoft Academic Graph (MAG) Fields of Study (FoS) taxonomy: A vast, multidisciplinary taxonomy with approximately 720,000 concepts structured in a six-level hierarchy.
- MeSH (Medical Subject Headings) controlled vocabulary: A biomedical taxonomy with approximately 29,000 topical descriptor tags.
- BioASQ competition dataset: Used for the MeSH taxonomy case study, containing annotated articles from PubMed.
Key Insights and Ideas
- Popularity isn't everything: The number of publications a concept is associated with (a proxy for popularity) does not necessarily equate to its true importance or influence within a taxonomy. Structural and topological properties play a significant role.
- Hierarchy matters: A concept's position within the taxonomic hierarchy (its level and depth) significantly impacts its dominance.
- Modular and extensible metric: The "Dominance Metric" is designed to be adaptable, allowing for the incorporation of new factors relevant to specific domains or taxonomies.
- Cleansing is crucial: Large, automatically generated taxonomies often contain noise (duplicates, erroneous relations) that requires manual or semi-automated cleansing to improve their representational quality.
- Bridging data silos: By creating compact, dominant versions of large taxonomies, the authors aim to provide a common semantic framework that can reduce reliance on proprietary systems and facilitate better content discoverability and analysis across publishers.
Key Results
- The proposed "Dominance Metric" was applied to the MAG FoS and MeSH taxonomies, resulting in "Dominant FoS" (DoFoS) and "Dominant MeSH" (DoMeSH) taxonomies.
- The DoFoS was created with approximately 200,000 concepts, representing a more compact and refined version of the MAG FoS.
- The DoMeSH was derived from the MeSH taxonomy, focusing on the most important topical descriptor concepts.
- The cleansing process for DoFoS involved identifying and merging approximately 4,300 duplicate concepts and refining hierarchical relations.
- Inter-Annotator Agreement (IAA) using Cohen's Kappa indicated substantial agreement among human curators for both deduplication and hierarchy refinement tasks, suggesting the reliability of the cleansing process.
Limitations and Future Work
- Manual Cleansing: While the paper uses semi-automated methods (Levenshtein distance, Cosine Similarity), the cleansing process still relies on significant human intervention from curators.
- Future Work: The authors plan to:
- Further expand the "Dominance Metric" by incorporating additional factors like semantic similarity and "trendiness" of tags.
- Develop methodologies for automatic identification of duplicate concepts and erroneous hierarchical relations to reduce manual effort.
- Compare their methodology against graph centrality metrics for identifying influential entities.
Summary updated