Proceedings of the 1st Workshop on Understanding Implicit and Underspecified Language

Murathan Kurfalı, Robert Östling, Marie Bexte, Andrea Horbach, Torsten Zesch, Michael Talita, Rani Anthonio, Weicheng Ma, Ruibo Liu, Lili Wang, Soroush Vosoughi, Ahmed Ruby, Christian Hardmeier, Sara Stymne, Östling Robert, Stymne Sara, Van Durme, Roy, Elhadad, Michael, Christian, Kurfal, stling Robert, Benjamin, Soroush, Lili, Wiriyathammabhum, Torsten, Hongxiao Bai, Hai Zhao, Chlo Braud, Pascal Denis, Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, Hsin-Ping Huang, Junyi Jessy, Li, Yangfeng Ji, Gongbo Zhang, Jacob Eisenstein, Najoung Kim, Song Feng, Chulaka Gunasekara, Luis Lastras, Man Lan, Jianxiang Wang, Yuanbin Wu, Zheng-Yu Niu, Haifeng Wang, Yang Liu, Xiaodong Zhang, Zhifang Sui, Daniel Marcu, Abdessamad Echihabi, Allen Nie, Erin Bennett, Noah Goodman, Emily Pitler, Annie Louis, Ani Nenkova, Talita Anthonio, Irshad Bhat, Michael Roth, Franck Khalil Mrini, Dernoncourt, Hung Quan, Trung Tran, Walter Bui, Ndapa Chang, Nakashole, Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, ukasz Kaiser, Illia Polosukhin, Beata Beigman Klebanov, Chee Leong, E GutierrezOpenalex Unknown 2021 · DOI · OpenAlex

Abstract

In implicit discourse relation classification, the goal is to predict the logical relationship between adjacent sentences in the absence of overt discourse connectives. This task is notoriously difficult for both humans and machines, leading to a shortage of high-quality annotated data. This paper presents a novel distant supervision pipeline that leverages the latent knowledge within large pre-trained language models to predict missing connectives, effectively transforming implicit relations into explicit ones. The resulting model achieves state-of-the-art performance in unsupervised settings and demonstrates remarkable cross-domain robustness.

Introduction

Discourse relations—the logical links that hold a text together—are essential for natural language understanding. While some relations are "explicit," signaled by connectives such as because, but, or however, many are "implicit." For instance, in the sequence "It's raining. I'm staying home," a causal link is understood despite the lack of a formal connective.

Identifying these implicit relations is a major challenge in Natural Language Processing (NLP). The subjectivity of manual annotation has led to a data bottleneck, where supervised models struggle due to a lack of labeled training examples. This research addresses this bottleneck by introducing a method that requires no manually labeled implicit data, relying instead on "distant supervision" through connective prediction.

Methodology: The Explicitation Pipeline

The authors propose a two-step pipeline that simplifies the classification of implicit relations by leveraging the abundance of explicit discourse data.

1. Explicitation via Language Modeling

Given a pair of sentences, the model first generates several "explicit" versions by inserting various one-word connectives (e.g., and, since, but). A large pre-trained language model (LM), such as RoBERTa-large or GPT-2 XL, is then used to score these candidates. The LM determines which connective fits most naturally within the specific context. This step effectively uses the LM as a bridge, translating the implicit problem into an explicit format.

2. Explicit Relation Classification

The highest-scoring connective (or a weighted average of all candidates) is then passed to a classifier. Crucially, this classifier is trained exclusively on explicit discourse relations, which are plentiful and easy to extract from large corpora. Because the classifier is highly accurate at identifying relations when a connective is present, the success of the system depends on the LM's ability to provide a meaningful "explicitation."

Data and Evaluation

The model was evaluated using two primary datasets:

  • PDTB 2.0 (Penn Discourse Treebank): The standard benchmark for discourse analysis. The model was trained on explicit relations and tested on implicit relations (Sections 21-22).
  • BioDRB (Biomedical Discourse Relation Bank): A specialized dataset used for cross-domain evaluation. This tested whether a model trained on news text (PDTB) could generalize to the complex logic of biomedical research articles.

Key Insights

  • Latent Discourse Knowledge: A significant finding is that modern LMs already possess a deep understanding of sentence-level logic from their general pre-training. They can often predict the correct missing connective without any task-specific fine-tuning.
  • The Power of Marginalization: Instead of relying on a single "best" connective, the authors found that averaging (marginalizing) over the entire distribution of possible connectives improved robustness, especially in cases where the top-ranked choice was ambiguous.
  • Comparison Relations: The model showed exceptional improvement in "Comparison" relations (e.g., contrast). This suggests that LMs are particularly sensitive to the linguistic cues that signal shifts in perspective or contradictions.

Results and Performance

The proposed method established a new state-of-the-art for unsupervised discourse classification:

  • Benchmark Success: The model achieved a top 4-way classification F-score of 41.10 using GPT2-large, outperforming previous unsupervised benchmarks.
  • Contrastive Logic Gains: For "Comparison" relations, the F-score nearly doubled from ~25% in previous work to 49.52%, significantly narrowing the gap with fully supervised models (58.35%).
  • Cross-Domain Robustness: The model maintained strong performance on the BioDRB dataset without additional tuning. It even outperformed some supervised models in zero-shot scenarios, proving that the method captures fundamental linguistic logic rather than just domain-specific patterns.

Limitations and Future Directions

While successful, the approach is currently limited to 65 one-word connectives, potentially missing the nuance provided by multi-word markers like "on the other hand." Furthermore, the system's performance is inherently capped by the quality of the underlying language model.

Future research will explore expanding the connective list and using this explicitation task as a diagnostic tool to evaluate how well new language models understand context. Additionally, the authors suggest this pipeline could be used to generate high-quality synthetic data to bolster supervised learning in the future.

Summary updated

Abstract

In implicit discourse relation classification, we want to predict the relation between adjacent sentences in the absence of any overt discourse connectives. This is challenging even for humans, leading to shortage of annotated data, a fact that makes the task even more difficult for supervised machine learning approaches.
BibTeX
@inproceedings{kurfal2021proceedings,
  author = {Murathan Kurfalı and Robert Östling and Marie Bexte and Andrea Horbach and Torsten Zesch and Michael Talita and Rani Anthonio and Weicheng Ma and Ruibo Liu and Lili Wang and Soroush Vosoughi and Ahmed Ruby and Christian Hardmeier and Sara Stymne and Östling Robert and Stymne Sara and Van Durme and Roy and Elhadad and Michael and Hardmeier and Christian and Kurfal and Liu and stling Robert and Stymne Sara and Van Durme and Benjamin and Vosoughi and Soroush and Wang and Lili and Wiriyathammabhum and Zesch and Torsten and Hongxiao Bai and Hai Zhao and Chlo Braud and Pascal Denis and Jacob Devlin and Ming-Wei Chang and Kenton Lee and Kristina Toutanova and Hsin-Ping Huang and Junyi Jessy and Li and Yangfeng Ji and Gongbo Zhang and Jacob Eisenstein and Najoung Kim and Song Feng and Chulaka Gunasekara and Luis Lastras and Man Lan and Jianxiang Wang and Yuanbin Wu and Zheng-Yu Niu and Haifeng Wang and Yang Liu and Sujian Li and Xiaodong Zhang and Zhifang Sui and Daniel Marcu and Abdessamad Echihabi and Allen Nie and Erin Bennett and Noah Goodman and Emily Pitler and Annie Louis and Ani Nenkova and Talita Anthonio and Irshad Bhat and Michael Roth and Talita Anthonio and Michael Roth and Irshad Bhat and Talita Anthonio and Michael Roth and Franck Khalil Mrini and Dernoncourt and Hung Quan and Trung Tran and Walter Bui and Ndapa Chang and Nakashole and Ahmed Ruby and Christian Hardmeier and Sara Stymne and Ashish Vaswani and Noam Shazeer and Niki Parmar and Jakob Uszkoreit and Llion Jones and Aidan Gomez and ukasz Kaiser and Illia Polosukhin and Peratham Wiriyathammabhum and Beata Beigman Klebanov and Chee Leong and E Gutierrez},
  title = {{Proceedings of the 1st Workshop on Understanding Implicit and Underspecified Language}},
  booktitle = {Openalex Unknown},
  year = {2021},
  doi = {10.18653/v1/2021.unimplicit-1},
}
Figure