The Problem of Idiosyncratic Word Combinations
Why do we say heavy rain but intense heat? Why do we pay attention but make an effort? To a native speaker, these choices feel automatic, but to a computer, they represent a fundamental challenge in natural language processing. These word pairings are known as collocations—combinations of words that appear together not purely because of logical compositionality, but due to idiosyncratic linguistic convention.
As Ulrich Held et al. observe in Collocations in Multilingual Generation, a collocation is far more than a random statistical co-occurrence. It is a structured relationship consisting of a determined partner (the base word, such as the noun attention) and a determining partner (the collocate, such as the verb pay).
For computational systems, simple word-to-word translations or statistical mappings fall short when dealing with these structures. If a machine translation system translates heavy smoker literally into French, it might produce fumeur lourd, which sounds highly unnatural to a native speaker; the correct phrase is grand fumeur (literally, big smoker). To solve this, computer scientists and linguists need a way to formalize the abstract semantic relationship that links a base word to its collocate, independent of the surface-level words used in any single language.
Lexical Functions as Semantic Operators
To capture these deep linguistic connections, researchers leverage the framework of Lexical Functions (LFs), a concept originating from Igor Mel'čuk's Meaning-Text Theory.
Think of a Lexical Function as a mathematical or logical operator. You provide it a base word (the keyword), and it outputs the grammatically and idiomatically correct collocate that fulfills a specific semantic role. For example, consider the abstract function $Magn$, which represents intensification ("very" or "to a high degree"):
$$Magn(\text{smoker}) = \text{heavy}$$ $$Magn(\text{rain}) = \text{heavy}$$ $$Magn(\text{heat}) = \text{intense}$$ $$Magn(\text{fumeur}) = \text{grand} \quad \text{(in French)}$$
By abstracting the relationship away from the surface words, LFs provide a systematic typology for word interactions. As explained by Marcos García et al. in Pay Attention when you Pay the Bills. A Multilingual Corpus with Dependency-based and Semantic Annotation of Collocations, these abstract labels allow systems to distinguish between different functional roles. This includes intensification ($Magn$) as well as the selection of "light verbs" used to perform the action of a noun (such as $Oper$, where $Oper(\text{attention}) = \text{pay}$).
This functional approach moves computational lexicons beyond simple static word pairings to capture deeper, dynamic linguistic dependencies—a goal championed early on by Raoul N. Smith et al. in An English Dictionary for Computerized Syntactic and Semantic Processing Systems.
Embedding Lexical Functions into Formal Grammars
How do we implement these mathematical-like operators inside a computer's parsing and generation engines?
One powerful method is to embed lexical functions directly into formal grammar frameworks. Dirk Heylen et al. demonstrate this in A Constraint-based Representation Scheme of Collocational Structures by treating LFs as relation names within a Head-driven Phrase Structure Grammar (HPSG). Instead of treating collocates as entirely independent words, the authors assign them partial lexical entries that generalize over their free-occurring forms. The lexicon explicitly links these collocates to specific base words via the LF relation name, creating a clean, interpretable architecture for encoding dependencies.
Similarly, in Lexical Functions and Machine Translation, the authors show that collocations can be represented by embedding LFs within a lexicon's dedicated collocates field, allowing a grammar to resolve word combinations dynamically during text processing.
Base Word: rain
└── Collocates Field:
└── Relation: Magn ──> Value: heavy
Beyond phrasal collocations, the structural logic of LFs extends to word derivations and argument structures. For instance, Igor Boguslavsky in Argument structure of adverbial derivatives in Russian introduces a compact notation using indices attached to lexical functions. This notation maps how derived words inherit and fill the valency slots of their base nouns.
This computational mapping of how grammatical changes correlate with meaning shifts builds upon early foundational work by Karl Dieter Bunting in Empirical Investigation of German Word Derivation with the Aid of a Computer. Even at the sub-word level, as noted by Yanhua Guo in The Philosophical Relationship Between the Origin of English and Chinese Affixes and Their Word Format, functional semantic categorization allows systems to organize linguistic components into distinct types based on their conceptual roles, enabling the systematic construction of complex meanings.
Applications in Translation and Text Generation
The primary advantage of Lexical Functions in practical NLP applications is their ability to serve as an interlingual bridge. Because an LF represents an abstract semantic relationship, it remains constant even when the surface words change across languages.
In machine translation, this abstraction eliminates the need for exhaustive bilingual phrase tables. As Igor Boguslavsky et al. show in Creating a Universal Networking Language Module within an Advanced NLP System, integrating LFs into an NLP environment allows the system to perform nuanced synonymous paraphrasing and make precise lexical choices when translating complex expressions between highly divergent languages.
[Source Text] "heavy rain"
│
▼ (Parse base & identify LF)
Base: rain, LF: Magn
│
▼ (Interlingual Bridge)
Base: pluie, LF: Magn
│
▼ (Generate target collocate)
[Target Text] "pluie battante"
In natural language generation (NLG), LFs provide fine-grained control over text coherence and coreference. Agnès Tutin et al. demonstrate in Lexical choice in context: generating procedural texts how LFs bridge the gap between underlying conceptual representations and language-specific expressions. For example, functions like Gener (generic word) and Syn (synonym) can be used to manage coreference naturally in a generated recipe, allowing the system to refer back to lamb as meat without sounding repetitive.
At a larger scale, Leo Wanner in On Lexically Biased Discourse Organization in Text Generation demonstrates that discourse structure relations can be modeled as sequences of LFs. By leveraging these functions, text generation systems can use reiteration and collocation as formal mechanisms to guide the thematic flow and organization of an entire text.
From Expert Rules to Deep Learning Evaluation
While Lexical Functions were originally developed as a tool for expert-engineered lexicons, they play a vital role in the era of deep learning and large language models (LLMs). They provide a "gold standard" framework for training models and evaluating whether neural networks truly understand the underlying mechanics of language or are merely memorizing surface patterns.
To scale this framework for modern NLP, Beatriz Fisas et al. developed CollFrEn, detailed in CollFrEn: Rich Bilingual English–French Collocation Resource. This resource provides thousands of collocations enriched with semantic categories and vector representations, bridging traditional lexicography with distributed embeddings.
This data allows researchers to probe the boundaries of modern neural architectures. In Evaluating language models for the retrieval and categorization of lexical collocations, Luis Espinosa Anke et al. framed collocation categorization as a sentence-pair classification task, fine-tuning language models like BERT to assign specific linguistic labels to word pairs within a sentence.
Their findings revealed an important insight: while modern language models excel at recognizing common statistical patterns, they frequently struggle with highly idiosyncratic collocations and subtle syntactic variations. This underscores the ongoing relevance of Lexical Functions. They offer a precise, structured blueprint of human semantic composition, serving as an essential tool for evaluating and improving how machines understand the idiomatic fabric of human language.
Key Foundational Work
An English Dictionary for Computerized Syntactic and Semantic Processing Systems (1973) — contributed 1 concept elements to this direction.
Empirical Investigation of German Word Derivation with the Aid of a Computer (1969) — contributed 1 concept elements to this direction.