TL;DR
This paper introduces Noun2Verb, a probabilistic framework that uses frame semantics to model how humans creatively convert nouns into verbs (e.g., "to Google"). It demonstrates that a generative, frame-based approach significantly outperforms discriminative models in interpreting and generating these novel word usages, bridging a gap in current NLP capabilities.
What Problem Does This Paper Address?
Humans have a remarkable ability to create and understand new word usages, particularly converting nouns into verbs (denominal verbs), like "to Google a search." While linguists have studied this phenomenon extensively, existing Natural Language Processing (NLP) systems struggle to interpret and generate these novel forms. Current models often fail to grasp the nuanced meaning derived from context and shared human knowledge, leading to incorrect interpretations or nonsensical generations. This paper aims to bridge this gap by developing a computational model that captures the underlying semantic and pragmatic principles humans use for this creative linguistic task.
Methods
The core of the Noun2Verb framework is a probabilistic model grounded in frame semantics. This approach views word meaning as tied to structured mental representations of events, entities, and their participants (semantic frames). The authors propose a dual problem: a listener module for comprehension (interpreting novel denominal verbs) and a speaker module for production (generating them).
They present three probabilistic models of increasing complexity:
- Discriminative Model: A baseline using state-of-the-art transformer models (like BERT) to predict interpretations and generations independently, without explicit collaboration or frame element representation.
- Partial Generative Model: Introduces listener-speaker collaboration and shared knowledge but omits explicit semantic frame elements. It uses variational inference for semi-supervised learning, allowing it to learn from both labeled and unlabeled data.
- Full Generative Model: The most advanced model, incorporating listener-speaker collaboration, shared knowledge, and explicit representation of latent semantic frame elements. This model aims to capture finer-grained semantic distinctions.
The models are trained to maximize the joint probability of utterances and their meanings, operationalizing the idea that successful communication relies on shared knowledge.
Datasets
The authors compiled and utilized three primary datasets for training and evaluation:
- DENOM-ENG: Denominal verb usages from contemporary English adults and children, extracted from linguistic literature.
- DENOM-CHN: Denominal verb usages from contemporary Mandarin Chinese, also from linguistic studies.
- DENOM-HIST: Denominal verbs from historical English corpora, tracing their emergence over two centuries.
Additionally, they created a DENOM-AMT dataset through crowdsourcing (Amazon Mechanical Turks) to obtain high-quality human annotations for model evaluation. This dataset includes both corpus-generated and human-generated paraphrases.
Key Insights and Ideas
- Frame Semantics is Crucial: The paper strongly suggests that understanding denominal verbs requires more than just surface-level linguistic patterns; it necessitates capturing underlying semantic frames and their constituent roles.
- Generative Models Outperform Discriminative Ones: For this task, generative models that explicitly model the interaction and shared knowledge between speaker and listener are significantly better than discriminative models, even those based on powerful transformers like BERT.
- Shared Knowledge is Key to Creativity: The success of human language users in creating and understanding novel denominal verbs hinges on shared contextual and encyclopedic knowledge, which the generative framework aims to model.
- Latent Frame Elements Capture Nuance: The full generative model's ability to infer latent frame elements allows it to distinguish between similar denominal usages that might otherwise be confused, especially when semantic relations are subtle.
- Cross-Lingual and Historical Generalization: The framework shows promise in generalizing across languages (English and Mandarin) and historical periods, indicating a robust approach to word class conversion.
Key Results
- Superior Performance: The full generative model consistently outperformed the partial generative model and the discriminative BERT-based baseline across comprehension and production tasks on the DENOM-AMT dataset. For instance, in English comprehension, the full generative model achieved an AUC of 0.82, compared to 0.71 for the discriminative model.
- Better Alignment with Human Judgement: KL divergence scores showed that the generative models' output distributions were more aligned with human annotations than the discriminative model, especially for human-generated paraphrases.
- Handling Ambiguity: The generative models were better at handling the inherent ambiguity in denominal verb interpretations, assigning higher probabilities to multiple plausible paraphrases, unlike the discriminative model which tended to favor a single, often corpus-driven, interpretation.
- Historical Prediction: The framework successfully predicted the emergence of novel denominal verbs over time, demonstrating its utility in diachronic linguistic analysis.
- Cross-Lingual Success: The models showed good generalization to Mandarin Chinese, supporting the universality of the frame-semantic approach.
Limitations and Future Work
- Data Sparsity: The authors acknowledge that model generalization can be poor when training instances for specific semantic types are extremely sparse, particularly for less common relation types like "location out."
- Complex Reasoning: The framework currently struggles with denominal verbs requiring complex, chained reasoning (e.g., "bee the cereal" to mean "add honey to cereal"), as it relies on simpler paraphrase templates.
- Blocking and Preemption: The current model doesn't explicitly handle linguistic phenomena like "blocking" (where a conventional form prevents a novel one, e.g., "drive" vs. "car to work"). While they argue this is a limitation of knowing what not to say, they suggest future work could explore interpreting such pre-emptive cases.
- Future Directions: The authors suggest exploring the framework's generality to other forms of word class conversion and incorporating richer ontological knowledge to handle more complex denominal meanings. They also aim to develop systems that can interpret and generate pre-emptive expressions.
Summary updated