Getting Idioms into a Lexicon Based Parser's Head

Oliviero StockACL 1987 · DOI

Introduction

Natural Language Processing (NLP) has long struggled with the inherent ambiguity and syntactic flexibility of idioms. Traditional approaches often treat idioms as either rigid "giant words" or simple patterns to be swapped out before parsing. However, these methods fail to account for the linguistic reality of idioms—particularly in languages like Italian—where idiomatic expressions can undergo passivization, modification, or interruption. This paper, originally published in 1986, proposes a compositional approach within the WEDNESDAY 2 parser that balances syntactic flexibility with cognitive realism.

The WEDNESDAY 2 Architecture

The proposed solution is integrated into WEDNESDAY 2, a nondeterministic, lexicon-based parser. Unlike systems that rely on a separate idiom dictionary, WEDNESDAY 2 embeds idiomatic knowledge directly within the lexicon entries of "thread" words (usually the head verb).

The architecture of the WEDNESDAY 2 parser, illustrating the flow between the lexicon, morphological processor, and the chart.

As shown in Figure 1, the system utilizes a morphological processor and a chart-based parsing mechanism. The core of the parser's intelligence lies in its "impulses"—dynamic instructions within lexical entries that seek specific fillers (such as subjects or objects) to complete a constituent. Idioms are defined as more specific versions of these impulses.

A Compositional View of Idioms

The author argues for a compositional view where idioms are anchored to a specific word that bears the main syntactic load. Each idiomatic fragment is assigned a specific weight. The parser operates using a Threshold Activation mechanism:

  1. Background Monitoring: As the parser builds a literal interpretation, it tracks idiomatic fragments in a "dynamic activation table."
  2. Activation: When the cumulative weight of detected fragments exceeds a predefined threshold, the system triggers a new idiomatic process.
  3. Multiprocessing: Crucially, the system does not discard the literal interpretation. Instead, it branches into parallel processes. This allows the parser to handle "garden path" idioms where a sentence initially appears idiomatic but concludes literally.

Implementation and Results

The system was implemented in Interlisp-D on a Xerox 1186 workstation. It was evaluated using complex Italian sentences, chosen for their high degree of syntactic shuffling. For example, the idiom Prendere il toro per le corna ("To take the bull by the horns") was successfully parsed even when passivized or when the word order was altered.

A structural representation of an idiomatic parse generated by the WEDNESDAY 2 system, showing the connections between lexical predicates.

Figure 2 demonstrates the resulting parse structure for a complex sentence. The use of "sleeping edges"—potential structures that only enter the active search space when triggered—ensures that the chart remains efficient despite the parallel processing of literal and figurative meanings. This approach aligns with psycholinguistic theories suggesting that humans process literal and idiomatic meanings in parallel rather than choosing one upfront.

Conclusion

By treating idioms as complex syntactic structures rather than flat strings, the WEDNESDAY 2 parser achieves a high degree of flexibility. Its multiprocessing architecture mirrors human cognitive processes, allowing for graceful failure of idiomatic interpretations without compromising the literal analysis. While the original implementation relied on hand-coded weights, the underlying logic provides a robust framework for handling the "nightmare" of idiomatic language in flexible-word-order environments.

Summary updated

BibTeX
@inproceedings{stock1987getting,
  author = {Oliviero Stock},
  title = {{Getting Idioms into a Lexicon Based Parser's Head}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1987},
  doi = {10.3115/981175.981183},
}
Figure