In traditional machine learning, we often treat categories as a flat list. In a "flat" world, the distance between a "Golden Retriever" and a "Labrador" is the same as the distance between a "Golden Retriever" and a "Toaster"—they are simply different labels. However, real-world knowledge is almost always organized into hierarchies or taxonomies, such as WordNet for language or Gene Ontology for biology.
Research into hierarchical semantic similarity seeks to move beyond binary "match or mismatch" logic. Instead, it treats the distance between concepts as a function of their position within a tree, recognizing that some differences are semantically "closer" than others.
The Geometry of Specificity
The most intuitive way to measure similarity in a hierarchy is to count the steps (path length) between two nodes. However, as Wanyin Li et al. argue, simple edge-counting fails to capture human perception. In a taxonomy, nodes located deeper in the tree represent more specific, concrete concepts. A one-step difference between two highly specific species of birds feels much smaller than a one-step difference between "Animal" and "Plant" at the root of the tree.
To account for this, researchers like Yangyang Wu et al. and Heyan Huang et al. propose multi-factor similarity functions. These models typically combine three elements:
- Path Length: The shortest distance between two nodes.
- Depth: The distance from the root to the nodes.
- Density: The number of sibling nodes in a specific branch.
By using nonlinear exponential functions to weight these factors, these metrics ensure that similarity increases with hierarchical depth. As Rui Yang et al. demonstrate in recommendation systems, matches at the most granular level (e.g., a specific storefront) should carry exponentially more weight than matches at a broad category level (e.g., "Shopping Mall").
The Role of the Lowest Common Ancestor
A central concept in this field is the Lowest Common Ancestor (LCA)—the most specific node that two concepts share. Alistair Kennedy et al. show that the LCA serves as a structural anchor for measuring relatedness. If two terms share a very specific LCA deep in the hierarchy, they are likely highly related.
However, not all ancestors are equally informative. Bo Li et al. introduced the Information Coefficient ($Sim_{IC}$) to address the "noise" of vague, high-level annotations. Their approach weights similarity based on the Information Content of the LCA—a measure of how rare or specific that shared ancestor is. This prevents two proteins from appearing similar just because they both share a generic, high-level label like "Biological Process."
Applications in Representation Learning
Integrating these hierarchical distances into modern AI models allows them to "understand" the relationships between the classes they are learning. Siqi Zeng et al. proposed using the Cophenetic Correlation Coefficient (CPCC) as a regularizer. This forces a neural network’s internal latent space to mirror the structure of a domain taxonomy, leading to better generalization when the model encounters unseen sub-classes.
Similarly, in the field of data retrieval, Long-Kai Huang et al. developed Class-Wise Supervised Hashing (CSH). By embedding taxonomic path lengths into the similarity matrix used for training, they ensure that the resulting binary codes preserve fine-grained semantic distances, making search results more relevant.
Beyond Simple Trees
While many models assume a clean tree structure, real-world data is often messier. Arnab Bhattacharya et al. explored using Earth Mover’s Distance (EMD) for objects annotated with multiple terms from an ontology. EMD treats similarity as a transport problem, calculating the "cost" of moving from one set of hierarchical annotations to another. This provides a more nuanced ranking than simple averages, especially when dealing with complex, multi-faceted entities.
In biological surveillance, the authors of a 2023 study on SARS-CoV-2 variants used a combination of Jaccard and Simpson similarity to detect parent-child relationships in evolving mutation networks. This allowed them to identify emerging variants by their hierarchical containment patterns weeks before traditional phylogenetic methods could.
By treating similarity as a structural property of a hierarchy rather than a flat distance, these methods allow systems to mimic human-like reasoning—recognizing that while all things are different, some differences are far more significant than others.