Linguistic typology classifies the world's languages based on their structural and functional properties—such as word order, phonological inventories, or morphological systems. For multilingual Natural Language Processing (NLP), these features act as a "structural DNA" that can guide cross-lingual transfer, allowing models to generalize from high-resource languages to low-resource ones (Edoardo Maria Ponti et al.).
However, building a complete map of global language features faces a massive bottleneck: data sparsity. As Helen O'Horan et al. point out, expert-curated typological databases like the World Atlas of Language Structures (WALS) are riddled with missing entries. Documenting thousands of languages requires extensive manual fieldwork, leading to a severe "bibliographic bias" where endangered or less-studied languages lack documented structural traits (Esther Ploeger et al.). This patchiness restricts our ability to build robust multilingual systems or accurately evaluate how well models generalize globally (Arturo Oncevay et al.; Roi Reichart et al.).
To overcome this, researchers have turned to computational methods to predict these missing features, treating the sparse typological matrix not as a broken dataset, but as an inference problem.
Typology as a Collaborative Filtering Problem
The key intuition behind predicting missing typological features is that linguistic traits do not vary randomly. They are deeply interconnected through universal grammar constraints and historical relationships. For example, if a language places its verbs before objects (VO word order), it is statistically far more likely to use prepositions rather than postpositions.
Xia Lu demonstrated that we can formalize these interactions using Probabilistic Graphical Models (PGMs). Instead of looking at isolated pairs of features, PGMs treat language typology as a web of conditional dependencies, allowing researchers to infer missing traits based on a language's known features. Johannes Bjerva et al. extended this approach by using tree-structured graphical models to automatically uncover complex, probabilistic implications hidden within sparse databases.
If languages and their features behave like users and preferences, we can fill the gaps using the same algorithms that power modern recommendation engines. The authors of A Probabilistic Generative Model of Linguistic Typology proved exactly that by framing typological prediction as a collaborative filtering problem. By applying matrix factorization to a sparse language-feature matrix, the model learns low-dimensional embeddings for both languages and features. Just as a streaming service predicts which movie you might like based on similar users, this generative model predicts missing grammatical features by identifying latent structural alignments across the world's languages.
Probing the Neural Fingerprint
What happens if a language is completely missing from a typological database, leaving no baseline features to bootstrap from? Computational linguists have discovered that we can bypass manual databases entirely by extracting typological features directly from neural network representations.
When a massively multilingual model is trained on text from hundreds of languages, it must internalize their underlying grammatical structures to succeed. Chaitanya Malaviya et al. demonstrated this by training a neural machine translation (NMT) system to translate over 1,000 languages into English. They found that the model naturally learns the structural DNA of these languages within its internal representations. By probing these learned embeddings, they could predict missing syntactic and phonological features in global databases with high accuracy.
This structural footprint is remarkably resilient. Yevgeni Berzak et al. showed that a native language (L1) leaves a distinct grammatical signature even when a speaker writes in a foreign language (L2). By analyzing the distribution of grammatical errors made by non-native English speakers, the authors successfully reconstructed the typological features and relationships of the writers' native languages—without ever exposing the model to a single word of the native language itself.
The Path to Complete Linguistic Profiles
Whether through matrix factorization or neural probing, filling the gaps in sparse typological data is crucial for the future of multilingual NLP. By moving away from rigid, manual curation and toward flexible, predictive frameworks, the field is transitioning from a patchy, English-centric view of language to a comprehensive, data-driven map of global linguistic diversity.
Key Foundational Work
Computational dialectology in Irish Gaelic (1995) — contributed 1 concept elements to this direction.
Joint Morphological-Lexical Language Modeling for Machine Translation (2007) — contributed 1 concept elements to this direction.