Systems that generate the inflected forms of a word, such as producing the Lithuanian genitive plural from a dictionary lemma, fail most often on words they have rarely or never seen. For those words the system usually has the grammatical tags it needs. What it is missing is knowledge of which inflection class the word belongs to, meaning which family of endings and stem changes it follows. Linguists describe these classes as a hierarchy. Broad default patterns sit at the top, narrower subclasses sit below them, and individually irregular words are the leaves. We propose to learn that hierarchy directly from inflection tables by placing every lexeme and every inflection pattern in hyperbolic space. This is a curved geometry whose volume grows exponentially with radius, so trees with many leaves fit in a few dimensions. In this space, general patterns settle near the center and idiosyncratic words drift toward the edge. A word-form generator conditioned on the learned position can then back off gracefully. A rare lemma whose class is uncertain lands near the center and receives the default endings. A lemma with distinctive evidence moves outward toward the specific subclass it resembles.
The proposal has three strands. Hyperbolic embedding provides the mechanism. Morphological realization, the task of producing surface forms from a lemma and a feature bundle, is the application. Standardized lexical representation constrains and validates the work: gold class labels from curated lexicons are used to check the hierarchy we discover, and the discovered hierarchy is exported as a linked-data paradigm graph so that it can be inspected rather than buried in a vector. The central claim can be falsified. At equal dimension, the hyperbolic hierarchy should improve accuracy on lemmas never seen in training, and the improvement should be largest when only a few cells of a word's paradigm are observed.
Why this intersection, and why not the obvious version
The obvious combination of these three directions is to embed the hierarchical feature schema of UniMorph 4.0 in hyperbolic space. That schema moved from flat tag sets to nested structures for phenomena such as case stacking and polypersonal agreement. We judged this a weak bet. Feature trees are shallow and contain only a few hundred tags, and the advantage of hyperbolic geometry is documented for large, deep trees with exponentially many leaves. The headline result of Poincaré Embeddings is 82,114 WordNet nouns at MAP 0.823 in 5 dimensions, against 0.168 for a 200-dimensional Euclidean embedding. A Euclidean embedding of a few hundred tags with a structured composition rule would very likely match a hyperbolic one. We therefore set that version aside.
The hierarchy that does have many leaves and an exponential fan-out lives over lexemes. A language has thousands to millions of lexemes, organized into macroclasses, classes, subclasses and irregulars. This structure is rarely annotated, and it is exactly what fails on rare words. Rule-based Morphological Inflection Improves Neural Terminology Translation inflects dictionary lemmas inside translation. On rare Lithuanian terms, a transformer inflection module reached 15.4% accuracy while hand-written rules reached 77.4%. The rules succeeded because they encode class membership and defaults; the neural model had no notion of either. The SIGMORPHON 2020 shared task reports top systems above 90% mean accuracy across 90 languages, but families such as Tungusic and Southern Daly fall below 60%, and top results depended on data hallucination and transfer rather than on any representation of paradigm structure. These two papers motivate the work: they show the gap. The hyperbolic papers supply the method. Hyperbolic Entailment Cones provides directed, transitive partial-order embeddings. Learning Continuous Hierarchies in the Lorentz Model provides a numerically stable model of hyperbolic space and a precedent for discovering an unannotated hierarchy from co-occurrence data, namely the Enron organizational chart. None of these papers touches word-internal morphology, and the research file found no work that joins hyperbolic embedding with morphological realization.
Core Claim
Let a pattern be a cell-specific edit script, for example "genitive plural: replace final -as with -ų". Each lexeme uses a set of patterns, one per cell. A pattern used by many lexemes is general. A pattern used only by a word and its few relatives is specific. We turn this bipartite usage relation into a partial order: pattern $p$ dominates lexeme $\ell$ if $\ell$ uses $p$, and pattern $p$ dominates pattern $q$ if every lexeme using $q$ also uses $p$. We embed this order with hyperbolic entailment cones. Each point $u$ owns a cone whose aperture $\psi(u)$ shrinks as $u$ moves away from the origin. Training minimizes the penalty
$$E(u,v)=\max\big(0,\ \Xi(u,v)-\psi(u)\big),$$
where $\Xi(u,v)$ is the angle at $u$ between the ray pointing away from the origin and the geodesic toward $v$. The penalty is zero exactly when $v$ lies inside $u$'s cone, following the construction of Hyperbolic Entailment Cones. Under this construction, default patterns that cover many lexemes are forced toward the origin, where cones are wide. Irregular lexemes are forced toward the boundary, inside the narrow intersection of the cones of the patterns they use. The distance from the origin therefore becomes a continuous measure of how idiosyncratic a word's inflection is. The paths of cones from the center outward reproduce a class hierarchy that nobody annotated.
A word-form generator, a character transducer of the kind used in SIGMORPHON 2020, is then conditioned on the lexeme's point $z_\ell$ as well as on the lemma and tags. For a lemma never seen in training, a small encoder predicts $z_\ell$ from the lemma string and from whatever forms of it have been observed. The encoder is trained with a cone-consistency loss, so weak evidence yields a point near the origin and the default inflection. This is hierarchical backoff carried out by the geometry rather than by hand-coded rules.
Research Program
The first component is the embedding. For each language we extract edit-script patterns per cell from UniMorph 4.0 tables, build the pattern–lexeme order, and embed it with Lorentz-model entailment cones. The training schedule includes the low-learning-rate burn-in that the Poincaré work found necessary near the boundary. We fix dimension at 10, based on the Lorentz result that 10 dimensions beat 200-dimensional prior models on WordNet, and sweep curvature.
The second component is the conditioned generator. We use a character transformer, the SIGMORPHON 2020 baseline family, and inject $z_\ell$ through a hyperbolic-to-tangent-space map as an additional input. We also train the lemma-to-point encoder. To simulate the realistic sparse case, training paradigms are deliberately subsampled to $k$ observed cells per lexeme, so the encoder learns to place lexemes from partial evidence.
The third component is validation and export. Where curated lexicons carry inflection-class or category information, we compare the learned tree to it. Two sources serve this purpose. The first is the category factorization in Camel Morph MSA for Arabic. The second is the lemma and hypolemma structure of LiLa for Latin, where adjectival participles form a sub-entry under the verbal lemma. That sub-entry gives a small, directed test of whether the geometry places hypolemma behavior beneath its verbal parent. We then serialize the learned cone hierarchy as paradigms and inflection rules in the OntoLex-Morph vocabulary. That work demonstrated analysis and generation via SPARQL, but only on toy fragments of Italian and Maltese. Our export supplies a large, machine-induced population of such a graph, which can be checked symbolically.
This export is our answer to the concern raised in Linked Lexical Knowledge Bases, that continuous embeddings strip away traceable relations. Because the embedding is a partial order, it can be read back out as explicit triples. We will measure how much of the order survives that round trip.
Baselines
The naive reference point is the SIGMORPHON 2020 character transformer trained on the same lemma-disjoint splits with no lexeme information. This is the setting in which Rule-based Morphological Inflection reports 15.4% on rare Lithuanian terms. On the standard task, the strongest reported results are the top shared-task systems (uiuc-01-0, deepspin-02-1, CULing-01-0), which the shared task overview reports at over 90% mean accuracy across languages. The ETH Zürich system (SIGMORPHON 2020 Task 0 System Description: ETH Zürich Team) reports 80.9% average accuracy for an exact-inference model and 75.6% for its transformer. The standard splits are not organized around unseen lemmas, and we found no reported result on Lacuna for lemma-disjoint inflection accuracy on SIGMORPHON 2020 languages. We will therefore rerun the strongest openly available system, the synchronous bidirectional transformer of the Illinois submission, with its data hallucination, on our splits. It gets the same hyperparameter search budget as our model. Beating it means higher exact-match accuracy on unseen lemmas, averaged over three seeds, with the margin exceeding seed variance. We will also report results on the original splits so that any regression on frequent words is visible.
For Lithuanian terminology, the rule-based inflector at 77.4% is the symbolic ceiling we aim to approach without hand-written rules. Closing most of the gap from 15.4% would be a strong result. Matching 77.4% would be the decisive one.
Two ablations isolate the key ingredient. The first keeps the cone objective and generator unchanged but replaces hyperbolic space with Euclidean space of the same dimension, i.e. Euclidean cones. This isolates curvature. The second keeps hyperbolic space but replaces cones with symmetric distance-based embedding. This isolates directionality, testing whether the asymmetric "pattern dominates lexeme" order matters or whether mere proximity suffices.
Why not just…?
Why not just give the generator a high-dimensional Euclidean lexeme embedding? This is the cheapest competitor, and it is the first ablation above. Prior evidence favors curvature on large trees: WordNet MAP of 0.823 at 5 hyperbolic dimensions versus 0.168 at 200 Euclidean dimensions in Poincaré Embeddings, and DOID F1 rising from about 0.43 to 0.82 in Hierarchical Mamba Meets Hyperbolic Geometry. But inflection-class trees may be shallower and less cleanly tree-shaped than WordNet. We will compare at 10 dimensions and also at 128 Euclidean dimensions. If the 128-dimensional Euclidean version matches us on unseen lemmas, the geometric claim is refuted for this domain and we will report that.
Why not just cluster lexemes by their edit-script signatures and feed the cluster ID? This is the most serious threat, because it is the classical symbolic route and it is cheap. Agglomerative clustering produces a tree, and the generator can condition on the path through it. Its weakness is that signatures are incomplete when only a few cells are observed, and a hard cluster assignment cannot express "probably class A, at least its parent". We will run it with the same generator across the cell-coverage sweep $k$. Our prediction is that it matches us at full coverage and falls behind as $k$ shrinks. If it does not fall behind, the hyperbolic machinery is unnecessary, and the contribution reduces to the observation that class conditioning helps.
Why not just add more hallucinated data or a bigger model? Hallucination was central to the top SIGMORPHON 2020 systems, and our strongest baseline includes it. It does not, however, tell the model which paradigm a novel lemma follows, and the Lithuanian terminology collapse occurred in exactly that regime. Combining hallucination with our conditioning is a cell of the experiment. If hallucination alone closes the unseen-lemma gap, we will conclude that lexeme structure is not the bottleneck.
Proposed Experiments
The data come from UniMorph 4.0 tables for languages with rich class systems and enough lexemes to form a real hierarchy: Lithuanian, Czech, Finnish, Latin, Arabic, and Russian if available. We also use the SIGMORPHON 2020 languages from families below 60%, to test the low-resource limit. We build lemma-disjoint splits and subsample each training lexeme to $k \in \{1,2,4,\text{all}\}$ observed cells. The Lithuanian terminology setup of Xu and Carpuat provides an external, downstream check.
Several checks are decisive. The first is unseen-lemma exact-match accuracy as a function of $k$, compared against all baselines; the expected signature is that our advantage grows as $k$ falls. The second is a rank correlation between a lexeme's distance from the origin and an irregularity score, defined as the fraction of its cells realized by minority patterns. A strong positive correlation is the mechanism-level evidence that the geometry encodes generality, analogous to the Enron norm–rank finding. The third is agreement between the learned tree and the gold category labels for Arabic and the conjugation and hypolemma structure for Latin, measured by tree-partition purity, with clustering as the comparison. The fourth is round-trip fidelity: the fraction of cone-implied dominance relations recovered after export to OntoLex-Morph and re-query via SPARQL.
We also anticipate a failure mode. Heteroclitic lexemes, which follow different classes in different cells, violate a single tree. We will report accuracy on them separately. If they degrade sharply, that marks where a tree-shaped geometry stops being the right prior, which is the limitation the Lorentz paper flags for non-hierarchical relations.
References
- UniMorph 4.0: Universal Morphology
- Poincaré Embeddings for Learning Hierarchical Representations
- Rule-based Morphological Inflection Improves Neural Terminology Translation
- SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection
- Hyperbolic Entailment Cones for Learning Hierarchical Embeddings
- Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry
- Camel Morph MSA: A Large-Scale Open-Source Morphological Analyzer for Modern Standard Arabic
- Harmonizing Divergent Lemmatization and Part-of-Speech Tagging Practices for Latin Participles through the LiLa Knowledge Base
- Linguistic LOD for Interoperable Morphological Description
- Linked Lexical Knowledge Bases: Foundations and Applications
- SIGMORPHON 2020 Task 0 System Description: ETH Zürich Team
- University of Illinois Submission to the SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection
- Hierarchical Mamba Meets Hyperbolic Geometry: A New Paradigm for Structured Language Embeddings