Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020

EMNLP 2020 · DBLP

Introduction

The field of Natural Language Processing (NLP) has experienced an unprecedented surge in research output over the last decade. While this growth signals a vibrant ecosystem, it has placed immense strain on traditional conference structures. The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) marked a pivotal moment in addressing this strain with the introduction of the Findings of the Association for Computational Linguistics (ACL). This inaugural volume represents a significant structural shift designed to capture high-quality research that would otherwise be lost to the "reject-and-resubmit" cycle.

The Scalability Crisis: Reviewer Fatigue and Publication Delays

The Program Committee Co-Chairs of EMNLP 2020 identified several critical failures in the traditional "binary" acceptance model. As submission numbers skyrocketed—reaching 3,359 for this cycle—acceptance rates plummeted. This created a "lottery effect," where many scientifically sound and valuable papers were rejected due to a lack of presentation slots rather than a lack of merit.

This system imposed three primary costs on the community:

  1. Reviewer Fatigue: The community spent immense effort reviewing papers that were eventually rejected and then resubmitted to the next conference, starting the cycle over.
  2. Publication Delay: Significant research was often delayed by months or even years as it bounced between venues.
  3. Loss of Nuance: The traditional "Accept/Reject" model failed to distinguish between papers that were "flashy" or "groundbreaking" and those that were "robust" and "methodologically sound."

The "Findings" Model: A Third Path

To mitigate these issues, the Findings model was introduced as a third path. During the standard EMNLP 2020 review process, Senior Area Chairs (SACs) and Program Chairs (PCs) identified papers that reviewers found meritorious—lacking fundamental flaws in theory, experimentation, or ethics—but which fell just below the competitive cutoff for the main conference.

Authors of these papers were offered publication in Findings of ACL. These are full, peer-reviewed publications indexed in the ACL Anthology. While they are not granted a presentation slot at the main conference, the organizers implemented a "match-making" service to facilitate their presentation at associated workshops, ensuring the work still received specialized visibility and the "stamp of approval" of a major conference.

Quantitative Impact and Community Adoption

The scale of the EMNLP 2020 cycle illustrates the necessity of this new tier. Out of 3,359 total submissions:

  • 752 papers (22.4%) were accepted to the main conference.
  • 520 papers (15.5%) were accepted to Findings.

This resulted in an aggregate acceptance rate of 37.9%, effectively doubling the volume of research preserved in the permanent record compared to the traditional ~20% rate. Perhaps the most telling metric of the model's success was the community's response: 86% of authors offered a spot in Findings chose to accept the offer rather than withdraw their work. This high adoption rate suggests that the community values the "Findings" brand and the speed of publication.

Diverse Contributions: New Benchmarks and Datasets

As a proceedings volume, Findings of ACL: EMNLP 2020 encompasses a vast array of linguistic tasks and datasets. The volume highlights the field's move toward global and specialized NLP, including several notable new benchmarks:

  • Cross-lingual Benchmarks: WikiLingua for abstractive summarization and RELX for cross-lingual relation classification.
  • Regional Language Support: KorNLI and KorSTS for Korean, and FQuAD for French question answering.
  • Specialized Domains: PolicyQA, a dataset focused on the reading comprehension of privacy policies.

Conclusion and Future Outlook

The introduction of Findings represents a move toward a more sustainable and inclusive publication ecosystem. By focusing on the "robustness" of research, the ACL has created a venue for solid work that contributes to the incremental progress of the field. While the chairs acknowledge potential impacts on workshop submission volumes, the primary goal remains clear: to establish Findings as a highly-cited, prestigious venue that reduces the burden on reviewers and accelerates the dissemination of NLP research. As the community continues to grow, this hybrid model of visibility and rigorous peer review may become the standard for all major computational linguistics conferences.

Summary updated

BibTeX
@inproceedings{2020findings,
  title = {{Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
}

Loading related content...

Figure