Knowledge-Augmented Frame Semantic Parsing with Hybrid Prompt-Tuning

Rui Zhang, Yajing Sun, Jingyuan Yang, Wei Peng

TL;DR

This paper introduces KAF-SPA, a novel architecture that enhances frame semantic parsing by integrating external knowledge from FrameNet into Pre-trained Language Models (PLMs). By using a hybrid prompt-tuning approach, KAF-SPA effectively disambiguates frame representations, leading to state-of-the-art performance on benchmark datasets.

What Problem Does This Paper Address?

Frame semantic parsing aims to identify the conceptual structure of a sentence, including the semantic frames evoked by words and their associated arguments. While Pre-trained Language Models (PLMs) have significantly improved performance in this area, they often struggle with disambiguating frame representations, especially when words have multiple meanings depending on context. This is because PLMs tend to rely heavily on co-occurrence patterns in their training data, which can lead to inaccurate frame predictions when those patterns don't align with the specific meaning intended in a given sentence. The challenge is to leverage the rich, structured knowledge embedded in semantic lexicons like FrameNet to guide PLMs towards more accurate frame disambiguation.

Methods

The proposed Knowledge-Augmented Frame Semantic Parsing Architecture (KAF-SPA) tackles this problem with two core modules:

  • Memory-based Knowledge Extraction Module (MKEM): This module acts as a knowledge retriever. It uses an end-to-end memory network to efficiently extract relevant frame definitions and role definitions from the FrameNet knowledge base. This extracted knowledge is then encoded into a continuous vector representation, serving as a "knowledge prompt."
  • Task-oriented Knowledge Probing Module (TKPM): This module integrates the knowledge extracted by MKEM into the PLM. It employs a novel "hybrid prompt-tuning" strategy, combining:
  • Continuous Prompts: The knowledge vector from MKEM is used to create a continuous prompt that injects semantic frame knowledge.
  • Discrete Prompts: Task-specific templates (e.g., "generate [frame_name]:") are used to guide the PLM towards specific subtasks like frame identification or argument identification.

This hybrid approach allows the model to adapt PLMs to the nuances of frame semantic parsing while simultaneously leveraging external knowledge. The entire system is trained end-to-end using a maximum likelihood objective, with an initial pre-training phase on exemplar instances from FrameNet to establish a better starting representation.

Datasets

The paper evaluates KAF-SPA on two standard benchmarks:

  • FrameNet1.5: Contains 1,019 frames and 9,634 frame elements.
  • FrameNet1.7: An extended version of FrameNet1.5, featuring 1,221 frames and 11,428 frame elements.

The authors use the same data splits as established in prior work (e.g., open-SESAME [10]).

Key Insights and Ideas

  • Knowledge as a Disambiguation Tool: The core insight is that structured semantic knowledge, like FrameNet definitions, can directly combat the ambiguity issues faced by PLMs in frame semantic parsing.
  • Hybrid Prompts for Targeted Adaptation: Combining continuous (knowledge-based) and discrete (task-specific) prompts is a novel way to guide PLMs, allowing them to both understand and adapt to the specific requirements of frame semantic parsing.
  • Memory Network for Efficient Knowledge Retrieval: Using a memory network for knowledge extraction is crucial for efficiency, preventing the model from being overwhelmed by the vastness of the FrameNet knowledge base.
  • Pre-training with Exemplars: Leveraging exemplar instances from FrameNet for initial pre-training helps the model develop a stronger foundational understanding of frames and roles before fine-tuning on the main task.

Key Results

KAF-SPA demonstrates significant improvements over strong baselines on both FrameNet1.5 and FrameNet1.7 datasets.

  • Frame Identification: The model achieves state-of-the-art results, outperforming baselines by a notable margin, especially on ambiguous test data (e.g., 86.6% vs. 82.3% accuracy on FrameNet1.5 Ambiguous test data).
  • Argument Identification: KAF-SPA also shows substantial gains, with improvements of over 3% F1 on FrameNet1.5 and over 5% F1 on FrameNet1.7 compared to the best baselines.
  • Ablation Studies: These studies confirm the effectiveness of both the MKEM and TKPM modules, showing performance drops when either is removed, highlighting their individual contributions.

Limitations and Future Work

  • Recall Performance on FrameNet1.5: The authors note that KAF-SPA exhibits slightly inferior recall performance on FrameNet1.5, potentially due to the dataset's imbalanced class distributions. They contrast this with the KID model, which achieves high recall but lower precision, suggesting it struggles to differentiate false positives.
  • Future Work: The authors plan to explore more effective ways to incorporate semantic information into the pre-training process for broader natural language understanding and reasoning tasks.

Summary updated

Abstract

Frame semantics-based approaches have been widely used in semantic parsing tasks and have become mainstream. It remains challenging to disambiguate frame representations evoked by target lexical units under different contexts. Pre-trained Language Models (PLMs) have been used in semantic parsing and significantly improve the accuracy of neural parsers. However, the PLMs-based approaches tend to favor collocated patterns presented in the training data, leading to inaccurate outcomes. The intuition here is to design a mechanism to optimally use knowledge captured in semantic frames in conjunction with PLMs to disambiguate frames. We propose a novel Knowledge-Augmented Frame Semantic Parsing Architecture (KAF-SPA) to enhance semantic representation by incorporating accurate frame knowledge into PLMs during frame semantic parsing. Specifically, a Memory-based Knowledge Extraction Module (MKEM) is devised to select accurate frame knowledge and construct the continuous templates in the high dimensional vector space. Moreover, we design a Task-oriented Knowledge Probing Module (TKPM) using hybrid prompts (in terms of continuous and discrete prompts) to incorporate the selected knowledge into the PLMs and adapt PLMs to the tasks of frame and argument identification. Experimental results on two public FrameNet datasets demonstrate that our method significantly outperforms strong baselines (by more than +3% in F1), achieving state-of-art results on the current benchmark. Ablation studies verify the effectiveness of KAF-SPA.
BibTeX
@misc{zhang2023knowledge,
  author = {Rui Zhang and Yajing Sun and Jingyuan Yang and Wei Peng},
  title = {{Knowledge-Augmented Frame Semantic Parsing with Hybrid Prompt-Tuning}},
  year = {2023},
  eprint = {2303.14375},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.14375},
}
Figure