For decades, the standard approach to understanding sentence structure across different languages relied on a strategy of abstraction. To teach a computer how to parse a new language with very little data, researchers would "delexicalize" the problem: they would strip away the actual words and look only at the Part-of-Speech (POS) tags. The intuition was that while words change between English and Spanish, the underlying grammar—how a "Noun" relates to a "Verb"—is a universal skeleton that can be transferred.
However, a growing body of research has demonstrated that this abstraction comes at a steep cost. By throwing away the words, we throw away the very cues that make language predictable.
The Information Loss of Abstraction
The fundamental problem with delexicalization is that POS tags are too coarse. As Wen Wang and Mary P. Harper argued in The SuperARV Language Model: Investigating the Effectiveness of Tightly Integrating Multiple Knowledge Sources, mapping specific words to abstract categories leads to a significant loss of word-specific information.
Consider the difference between the verbs "give" and "sleep." Both are tagged as verbs, but "give" requires a giver, a gift, and a receiver, while "sleep" only requires a sleeper. A delexicalized parser sees only "Verb" and loses the ability to use these "subcategorization" cues to resolve structural ambiguities. Yuval Krymolowski and Ido Dagan highlighted this in Incorporating Compositional Evidence in Memory-Based Partial Parsing, noting that excluding lexical information prevents models from using semantic cues that are essential for disambiguation.
Words Outperform Tags
Surprisingly, even in monolingual settings, the reliance on POS tags as a prerequisite for parsing has been called into question. In Shallow Parsing on the Basis of Words Only: A Case Study, Antal van den Bosch and Sabine Buchholz showed that once a training set reaches a modest size, a learner trained on raw words actually outperforms one trained on POS tags. Lexical information, they argue, is a more powerful predictor of phrase boundaries than abstract grammatical categories.
Bridging the Lexical Gap
If lexical information is so vital, how do we use it in cross-lingual transfer where the words in the source language (English) don't match the target language (Swahili)?
The breakthrough came from moving away from "delexicalized" transfer toward "lexicalized" transfer using shared vector spaces. Instead of ignoring words, researchers like Jiang Guo and Wanxiang Che developed methods to map different languages into a common embedding space. In Cross-lingual Dependency Parsing Based on Distributed Representations, they demonstrated that by using shared word embeddings, a parser can "see" that an English word and its foreign counterpart share a similar semantic and syntactic role, leading to a 10.9% reduction in error over delexicalized methods.
This theme was echoed by Long Duong and Trevor Cohn in A Neural Network Model for Low-Resource Universal Dependency Parsing. They proposed a joint neural architecture where a high-resource language "teaches" a low-resource one. By keeping embeddings language-specific but anchoring them with a bilingual dictionary, they retained the "lexical cues" that traditional delexicalization discarded.
The Modern Synthesis
The shift toward lexicalization has culminated in the use of massive multilingual pre-trained models. Ke Tran and Arianna Bisazza showed in Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations that models like multilingual BERT can perform zero-shot parsing precisely because they capture rich semantic information that delexicalized models miss.
Even the way we select which language to transfer from has become lexicalized. Robert Litschko and colleagues, in Towards Instance-Level Parser Selection for Cross-Lingual Transfer of Dependency Parsers, moved beyond picking a single "source language" for a "target language." Instead, they use Transformer-based models to pick the best parser for every individual sentence, leveraging the deep contextual features that only lexicalized models provide.
The consensus across two decades of research is clear: while abstraction helps us generalize, the "nuanced semantic relationships between specific words," as Jiang Guo puts it, are the true engine of accurate syntactic analysis. To understand the structure of a sentence, you cannot ignore the words themselves.