Processing Non-Compositional Semantics in Natural Language

925 concepts590 papers2 proposals

Most natural language processing systems operate on a fundamental premise: the principle of compositionality. This principle states that the meaning of a complex expression is a direct function of the meanings of its parts and the syntactic rules used to combine them. From early phrasal systems like the one introduced by Wilensky et al. in PHRAN - A Knowledge-Based Natural Language Understander to modern Transformer-based large language models, NLP architectures rely heavily on this bottom-up aggregation. Whether summing word vectors or computing multi-head self-attention over sub-word tokens, models assume that text can be understood by systematically combining smaller semantic units.

Where Compositionality Fails

Human language is filled with constructs that violate this assumption: non-compositional expressions. These include idioms ("kick the bucket," "cut corners"), metaphors, and certain multi-word expressions (MWEs) or compounds ("baby oil," "hot dog"). As Bhat et al. demonstrate in PIE: A Parallel Idiomatic Expression Corpus for Idiomatic Sentence Generation and Paraphrasing, the holistic meaning of a highly non-compositional idiom bears virtually no relation to its individual constituent words.

Why does this break modern architectures? Consider how a standard Transformer processes a phrase. Its attention mechanism computes contextualized representations by looking at individual tokens. For an arbitrary, culturally dependent idiom, there is no statistical or logical path from the token "bucket" to the concept of "death." As Stowe et al. observe in IMPLI: Investigating NLI Models' Performance on Figurative Language, while metaphors often retain some compositional logic, idioms represent an entirely arbitrary mapping. Purely additive or aggregative models, even when augmented with noise terms as explored by Cotterell et al. in Morphological Smoothing and Extrapolation of Word Embeddings, fundamentally struggle to map these holistic meanings accurately.

The Cost of Literalism

When a model encounters a non-compositional phrase without explicit figurative awareness, it defaults to a literal interpretation. In A Long Hard Look at MWEs in the Age of Language Models, Vered Shwartz points out that this literalism creates factually incorrect internal representations, leading to severe failures in downstream reasoning.

These failures cascade across various NLP applications:

Disambiguation and the Spectrum of Meaning

Compounding the challenge is the fact that many multi-word expressions are semantically ambiguous; they can be interpreted either literally or idiomatically depending on the context. The authors of Japanese Idiom Recognition: Drawing a Line between Literal and Idiomatic Meanings address this boundary, emphasizing the need for models to dynamically distinguish between these senses.

In fact, multi-word expressions exist on a continuous spectrum. In Verb Noun Construction MWE Token Classification, Mona T. Diab explains that MWEs range from completely transparent and compositional to entirely opaque and non-compositional. Interestingly, supervised models often find the literal usages of potentially idiomatic phrases harder to identify than their idiomatic counterparts, suggesting that literal expressions exhibit higher linguistic variability or require more sophisticated contextual features to isolate.

Strategies for Non-Compositional Awareness

To bridge this gap, researchers have developed several distinct strategies to make models "idiom-aware."

Dual Embeddings and Hybrid Representations

One approach is to explicitly model both compositional and non-compositional pathways. The authors of Adaptive Joint Learning of Compositional and Non-Compositional Phrase Embeddings developed a framework that adaptively blends representations by learning a "compositionality score" to decide whether a phrase should be treated as a single holistic unit or parsed word-by-word.

Similarly, for Chinese idiom (Chengyu) prediction, Jiang et al. utilize a dual embedding model in A BERT-based Dual Embedding Model for Chinese Idiom Prediction that splits representations into local grammatical features and global semantic meanings. Treating idioms as unique, holistic semantic units within a context allows models to vastly outperform standard word aggregations, as shown by Jiang in Learning and Evaluating Chinese Idiom Embeddings.

Contrastive Learning and Structural Separation

Rather than just blending vectors, some frameworks actively force the model to separate literal and figurative spaces. In CLCL: Non-compositional Expression Detection with Contrastive Learning and Curriculum Learning, Zhou et al. employ contrastive learning to pull literal and figurative representations apart in the embedding space, training on a curriculum of increasingly difficult examples to improve generalization to unseen expressions.

External Knowledge Graphs

When statistical patterns in text are insufficient, external commonsense knowledge can provide the missing link. Zeng et al. introduced IEKG: A Commonsense Knowledge Graph for Idiomatic Expressions, which maps explicit "if-then" reasoning to figurative meanings. This allows models to look up the abstract implications of an idiom (e.g., mapping "licking someone's boots" to "insincerity") rather than relying on standard compositional semantics.

Pre-processing and Paraphrasing

Another pragmatic direction involves "de-idiomatizing" text before it reaches downstream tasks. Sneha Mehta demonstrates in Simplify-Then-Translate: Automatic Preprocessing for Black-Box Translation that translating complex, non-compositional phrases into literal equivalents via an automatic preprocessing step significantly boosts the accuracy of black-box translation systems. This is supported by Zhou et al. in Idiomatic Expression Paraphrasing without Strong Supervision, who leverage dictionary definitions and iterative back-translation to substitute figurative expressions with literal paraphrases without requiring massive parallel corpora.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 592 papers
  2. Version 1 · 2026-10-04 · 705 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_semantic_non_compositionality_d523b8f8,
  title = {semantic non-compositionality},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-non-compositionality/rdr_d523b8f8b3494030a6d0ce73758b954b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 2, frozen text):

@misc{lacuna_semantic_non_compositionality_d523b8f8_v2,
  title = {semantic non-compositionality},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-non-compositionality/rdr_d523b8f8b3494030a6d0ce73758b954b/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake