Introduction
Derivational morphology is the linguistic process of forming new words from a root (e.g., happy → happiness). While this seems intuitive to humans, it presents significant challenges for computational models. Two primary hurdles exist: suffix ambiguity, where multiple suffixes might be plausible for a given transformation (e.g., for ground, should it be grounding or groundment?), and orthographic irregularity, where spelling changes are unpredictable (e.g., speak → speech).
Purely character-based sequence-to-sequence (seq2seq) models often struggle with these irregularities because they lack the semantic "common sense" to navigate non-predictable spelling patterns. This paper proposes a hybrid approach that bridges the gap between meaning and spelling by aggregating orthographic patterns with distributional semantics.
The Aggregation Model
The core contribution is the Aggregation Model (AGGR), which fuses two distinct modeling paradigms:
- The Orthographic Model (SEQ): A character-level seq2seq model using a bidirectional LSTM with attention. It excels at capturing regular orthographic patterns, such as the standard addition of "-ly" to form adverbs.
- The Distributional Model (DIST): A novel feed-forward network operating in word embedding space. It learns to map the vector of a root word to the vector of its derived form. By leveraging semantic embeddings, this model acts as a "semantic safety net" for irregular forms where spelling changes drastically but the semantic relationship remains consistent.
The final hypothesis is selected by a logistic regression classifier that decides which model to trust for a given input. The system is further enhanced by Dictionary Constraints (+DICT), which restrict the decoder to valid English words found in a large corpus, and Word Frequency Rescoring (+FREQ), which uses N-gram data to favor common words when the model is uncertain.
Experimental Setup
The models were evaluated using a dataset derived from NomBank, containing 4,222 training triples (root, transformation, derived word) across categories such as ADVERB, AGENT, and NOMINAL. The distributional model utilized pre-trained Google News Word2Vec vectors (929k words), while the dictionary and frequency components were informed by the Google Books Ngram corpus, filtered to a dictionary of approximately 360k words.
Results and Analysis
The aggregation model demonstrated superior performance over previous state-of-the-art (SOTA) systems across all metrics. In open-vocabulary settings, the AGGR model achieved 78.0% accuracy, a significant jump from the previous SOTA of 71.7%. When augmented with dictionary and frequency constraints (AGGR+FREQ+DICT), accuracy rose to 82.4%.
A key insight is the complementary nature of the components: SEQ handles high-frequency regular suffixes, while DIST improves performance on rare or irregular forms. The most dramatic gains were seen in NOMINAL transformations (e.g., pessimistic → pessimism), where the model achieved a 49% relative error reduction. This category is particularly difficult due to the high number of competing suffixes like -ity, -ness, and -ence.
Conclusion
By combining character-level sequence modeling with distributional semantics and corpus-based constraints, this research significantly advances the state-of-the-art in English derivational morphology. While the model is currently limited by its reliance on a pre-defined embedding dictionary and cannot easily generate neologisms, its success in handling complex orthographic jumps marks a major step forward in bridging the gap between meaning and spelling.
Summary updated