How do we determine how "informative" or "specific" a concept is? In information theory, a concept's information content (IC) is inversely proportional to its probability: rare concepts carry high information, while common concepts carry low information. Traditionally, natural language processing and bioinformatics have estimated this probability by counting term frequencies within massive text corpora or annotation databases. For instance, if the term "glucose-6-phosphate isomerase activity" appears rarely in literature compared to the broad term "metabolism", it is assigned a higher information content.
However, relying on external corpora introduces a fundamental flaw: your semantic similarity metrics become a slave to the biases, noise, and incompleteness of that specific text collection.
The core insight behind intrinsic information content is that we do not need an external corpus at all. Human experts have already painstakingly organized these concepts into structured hierarchical graphs, such as WordNet, ChEBI, or the Gene Ontology (GO). The topology of the graph itself—how many children a node has, how deep it sits, and how densely packed its neighborhood is—implicitly encodes its exact semantic specificity.
Translating Graph Topology into Specificity
To understand why this works, imagine a hierarchical tree. At the very top sits a root node representing a completely generic concept (e.g., "Entity" or "Biological Process"). Every other node in the graph is a descendant of this root. As you move down the hierarchy, the branches split into increasingly granular concepts.
Because human experts design these hierarchies systematically, the structural density surrounding a node serves as an accurate proxy for its meaning. We can calculate a concept's structural probability purely by looking at its cardinality—the count of the node itself plus all of its descendants.
[Root: Entity] <- High cardinality, high probability, LOW Information Content
/ \
[Object] [Concept]
/ \
[Vehicle] [Tool]
/
[Car] <- Zero descendants, low probability, HIGH Information Content
Xiang et al. and Nobani et al. formalize this mathematical intuition by defining a node's intrinsic probability as the ratio of its descendants to the total number of nodes in the entire hierarchy. A node near the bottom of the tree has very few or zero descendants, yielding a tiny structural probability and a high intrinsic information content.
This topological mapping extends beyond simple descendant counting:
- Asymmetric Relationships: Schickel et al. demonstrate that the intrinsic branching factor and depth of an ontology can be used to model asymmetric similarity. They show that generalizing from a specific concept to a broad one (moving up the tree) is fundamentally different from specializing downward, an intuition easily captured by graph topology but lost in flat corpus statistics.
- Multi-Feature Topology: Rather than looking at a single metric, Tian et al. combine node depth, ancestor counts, and descendant density to construct a highly nuanced and multi-faceted measure of intrinsic semantic weight.
Why Intrinsic Metrics Outperform External Corpora
Deriving semantic similarity exclusively from internal graph structures offers several distinct advantages over data-driven approaches.
1. Immunity to Data Bias and Research Trends
External text corpora and annotation databases are notoriously skewed by real-world research trends. Yu et al. point out that in biomedical datasets, certain popular proteins or diseases are heavily studied and over-represented, while others remain poorly understood or completely unannotated. Corpus-based metrics mistake this research popularity for semantic specificity. Intrinsic topology fixes this by relying on the stable, structural logic of the taxonomy rather than volatile statistical counts.
2. Stability and Portability
Because external databases are updated constantly as new literature is published, corpus-based similarity scores fluctuate over time. Qi demonstrates that calculating semantic similarity solely from the intrinsic structure of a knowledge graph ensures that similarity scores remain stable and reliable. Furthermore, Lamurias et al. emphasize that eliminating the dependency on massive, pre-indexed text libraries drastically improves system portability, allowing the same similarity algorithm to be deployed instantly across different domains without retraining.
Validation Across Domains
This structural approach has proven remarkably powerful across wildly different fields, frequently matching or outperforming complex, data-heavy models.
Natural Language Processing
In an era dominated by massive neural language models, Renner et al. showed that a simple, unsupervised method using the intrinsic structure of the decades-old WordNet hierarchy could outperform deep learning models at predicting graded lexical entailment (determining how strongly one word implies another). This proves that the density and depth of human-curated trees are inherently more reliable indicators of concept specificity than noisy statistical counts from large text datasets.
Bioinformatics and Cheminformatics
In biology, the Gene Ontology is often highly uneven; some branches are mapped out in extreme, granular detail, while others remain sparse. To handle this structural imbalance, Bader et al. developed a topological clustering approach that quantifies node specificity purely from graph hierarchy. By accounting for the uneven density of different branches without relying on external annotation frequencies, their method significantly outperformed traditional corpus-dependent benchmarks in identifying true protein-protein interactions.
Key Foundational Work
Combined measurement approach for semantic similarity of terms (2010) — contributed 2 concept elements to this direction.
OSS: A Semantic Similarity Function based on Hierarchical Ontologies (2007) — contributed 2 concept elements to this direction.
An improved method for scoring protein-protein interactions using semantic similarity within the Gene Ontology (2010) — contributed 1 concept elements to this direction.