Position Specific Character N Grams for Lexical Modeling

20 concepts13 papers1 proposal

A standard character n-gram model treats a word as a flat bag of sub-strings. However, in natural language, a sub-word sequence's position is often more important than its raw presence. For example, the sequence "ing" at the end of an English word strongly signals a gerund or participle, whereas the same sequence at the beginning of "ingot" is merely part of the root stem.

For tasks like morphological parsing, etymological classification, and name origin recognition, critical linguistic signals are highly localized—typically concentrated at prefixes, suffixes, or word boundaries. By making character features sensitive to their exact position within a word, machine learning models can capture deep morphological and syntactic structures without requiring complex, language-specific rules.

Anchoring to Word Boundaries via Terminal Markers

How do we give a flat n-gram model a sense of geography? The simplest yet most effective technique is appending a unique terminator character to the end of a word string.

As demonstrated by Dinu et al. in their study on the Romanian gender system, Dealing with the Grey Sheep of the Romanian Gender System, the Neuter, a terminal marker allows a classifier to assign distinct weights to character sequences based on whether they function as terminal suffixes or internal stems. Without this marker, a model cannot distinguish between a sequence occurring inside a root and the same sequence acting as an inflectional suffix.

In Learning How to Conjugate the Romanian Verb. Rules for Regular and Partially Irregular Verbs, Dinu et al. show that this implicit positional context is critical for identifying morphological boundaries, enabling models to accurately predict complex verb conjugations and internal stem changes (apophony).

Standard N-grams for "singing":  ["sin", "ing", "ngi", "gin", "ing"]  <- Duplicate "ing"
Boundary-Anchored N-grams:      ["sin", "ing", "ngi", "gin", "ing$"] <- Differentiates the suffix

Alternatively, models can explicitly incorporate a weighting factor based on a character's exact index within a word. Wong et al. utilized this mechanism in Adapting Multilingual Parsing Models to Sinica Treebank, demonstrating that position-sensitive character weighting allows a model to prioritize information from prefixes or suffixes when predicting the part-of-speech (POS) of unknown words.

Similarly, Zhang et al. showed in Name Origin Recognition Using Maximum Entropy Model and Diverse Features that position-specific n-grams capture structural patterns exclusive to the beginnings or ends of strings, providing the precise structural context needed to identify naming conventions across different languages.

Morphological and Syntactic Inference

Why are word endings so informative? In many inflectional languages, grammatical properties like gender and lexical class are phonologically and morphologically encoded in suffixes.

Nicholson et al. confirmed this asymmetry in Die Morphologie (f): Targeted Lexical Acquisition for Languages other than English, showing that character sequences at the end of words are significantly more reliable indicators of grammatical gender than those at the beginning. This principle underpins early text-to-speech pipelines as well; for example, the system designed by Pounder et al. in Morphological Analysis for a German Text-to-Speech System determines a word's lexical class by isolating and analyzing its final constituent morph.

Even in languages that lack explicit orthographic cues like capitalization, position-specific morphological features can step in. As explored by Gali et al. in Aggregating Machine Learning and Rule Based Heuristics for Named Entity Recognition, language-specific suffixes serve as highly effective proxies for identifying entity types (such as distinguishing locations from person names) in South Asian scripts where proper nouns are not visually capitalized.

Etymological Discrimination

The utility of position-specific features extends beyond grammar into historical linguistics, specifically in distinguishing between cognates (words sharing a common ancestral root) and borrowings (loanwords).

When a word is borrowed from one language into another, it rarely enters unchanged; it typically undergoes morphological adaptation to conform to the grammatical rules of the recipient language. This adaptation almost always occurs at the morphological boundaries. In Automatic Discrimination between Cognates and Borrowings, Ciobanu et al. demonstrate that analyzing prefixes and suffixes reveals these etymological origins by detecting the modification or removal of affixes. Conversely, true cognates tend to preserve or transform their affixes through consistent, historical sound laws.

By focusing on n-grams around alignment mismatches at the ends of words, the authors show in Automatic Detection of Cognates Using Orthographic Alignment that these localized character sequences provide a robust signal for identifying shared ancestry. This character-level positioning, combined with broader indicators like capitalization and language IDs, forms the backbone of high-performing supervised cognate detection systems, as demonstrated by Limisiewicz in ÚFAL Submission for SIGTYP Supervised Cognate Detection Task.

Scaling to Sentence-Level Localized Context

The intuition that linguistic transitions are highly localized at boundaries also applies at the sentence level, particularly in code-switching (the alternation between two languages in speech or text). Just as a word boundary dictates morphological changes, a language switch boundary is heavily influenced by the immediate, localized context preceding it.

Soto et al. discovered in The Role of Cognate Words, POS Tags and Entrainment in Code-Switching that cognate words immediately preceding a code-switch significantly facilitate the transition, acting as linguistic triggers, whereas cognates appearing after the switch have no such effect. This asymmetry highlights that the preceding structure primes the speaker for a switch.

Furthermore, Chi et al. proved in Analyzing the Role of Part-of-Speech in Code-Switching: A Corpus-Based Study that the statistical influence of part-of-speech tags on code-switching is a highly localized phenomenon. The predictive power of the syntactic context peaks precisely at the point of the language switch and rapidly decays as words move further away.

Recognizing these localized, boundary-specific features allows for significantly better performance in code-switched language modeling without relying on manual labeling, as shown by Soto et al. in Improving Code-Switched Language Modeling Performance Using Cognate Features.

Key Foundational Work

Die Morphologie (f): Targeted Lexical Acquisition for Languages other than English (2006) — contributed 1 concept elements to this direction.
Morphological Analysis for a German Text-to-Speech System (1986) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_position_specific_n_grams_95383edb,
  title = {position-specific n-grams},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/position-specific-n-grams/rdr_95383edb8966457080cec968c083a942},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_position_specific_n_grams_95383edb_v1,
  title = {position-specific n-grams},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/position-specific-n-grams/rdr_95383edb8966457080cec968c083a942/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake