Modal Dependency Parsing via Language Model Priming

Jiarui Yao, Nianwen Xue, Bonan MinNAACL 2022 · DOI

Introduction to Modal Dependency Parsing

In natural language processing, determining the "factuality" of a statement is a complex task. Information is rarely presented as absolute truth; instead, it is often filtered through various sources or "conceivers." For instance, in the sentence, "The governor confirmed that the fire was contained," the event (the fire being contained) is attributed to a specific source (the governor).

Modal Dependency Parsing (MDP) is the task of identifying these sources and the level of certainty they express. The output is typically a tree structure where the "AUTHOR" of the text serves as the root, and events are linked to their respective conceivers via edges representing modal strength (e.g., Full Positive, Partial Negative).

A Modal Dependency Tree structure showing the relationship between an author, a source (Jeroen Weimar), and events.

Limitations of Previous Approaches

Previous state-of-the-art models for MDP relied on a "bottom-up" ranking approach. These systems first extracted all potential events and conceivers as independent entities and then attempted to link them. This methodology suffered from two primary drawbacks:

  1. Error Propagation: If a model failed to recognize an entity as a potential conceiver during the initial extraction phase, it was impossible to correctly attribute an event to that source later.
  2. Context Blindness: An entity's role as a "conceiver" is functional and event-specific. Previous models often failed to capture the nuanced relationship between a specific event and the person reporting it.

Methodology: Prompt-Based Priming

To address these issues, this paper proposes a novel prompt-based priming method. Rather than attempting to resolve all dependencies simultaneously, the model focuses on one event at a time, leveraging the internal attention mechanisms of Pre-trained Language Models (PLMs).

The process begins with Event Extraction, using standard BIO tagging to identify event spans. For each identified event, the model constructs a specific input sequence (a prompt) for the PLM:

[CLS] <EVENT> event_word </EVENT> [SEP] context_sentences [SEP]

The prompt-based priming mechanism for conceiver tagging, illustrating how the model identifies sources for a specific event.

By "priming" the model with the event of interest, the PLM can more effectively tag the context sentences to identify the "parent" conceiver and, if necessary, the "grandparent" conceiver (the source of the source). The prompt also includes special tokens like <AUTHOR> and <NULL> at the end of the sequence, allowing the model to tag the "AUTHOR" as the conceiver if no other source is found in the text. Simultaneously, a separate neural head predicts the modal strength of the relationship.

Evaluation and Results

The researchers evaluated their approach on English news datasets and established the first benchmark for Chinese MDP using the Xinhua news dataset. The prompt-based model demonstrated significant improvements over previous ranking-based baselines:

  • English: Achieved a 2.6% absolute improvement in F-score over the previous state-of-the-art ranking model.
  • Chinese: Achieved a 4.6% absolute improvement over the baseline.

A key strength of the model is its robustness in cross-sentence accuracy. In news reporting, sources are frequently introduced in different sentences than the events they describe. The priming approach allows the model to maintain context across these boundaries more effectively than previous methods.

Challenges in Chinese MDP

While the model set a new state-of-the-art for Chinese MDP, a performance gap remains (63.6% F-score in Chinese vs. 71.9% in English). This is largely attributed to the "pro-drop" nature of the Chinese language, where pronouns (like "he" or "she") are frequently omitted when they can be inferred from context. Identifying these implicit or "dropped" conceivers remains a significant hurdle for future research.

Conclusion

By framing Modal Dependency Parsing as a prompt-based tagging task, this research demonstrates that language model priming can effectively overcome the limitations of traditional ranking models. This approach not only improves performance in English but also provides a robust foundation for MDP in pro-drop languages like Chinese.

Summary updated

BibTeX
@inproceedings{yao2022modal,
  author = {Jiarui Yao and Nianwen Xue and Bonan Min},
  title = {{Modal Dependency Parsing via Language Model Priming}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.naacl-main.211},
}
Figure