Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations

Heike Adel, Alan Akbik, Zeynep Akkalyoncu, Akari Asai, Eleftherios Avramidis, Wei Bi, Georgeta Bordea, Ari Bornstein, Aljoscha Burchardt, Eric Charton, Hai Leong Chieu, Simone Conia, D Croce, Montse Cuadros, Marina Danilevsky, Pradipto Das, Victor Dibia, Chenchen Ding, Carsten Eickhoff, Kshitij Fadnis, Nicolas Rodolfo Fauceglia, Anthony Ferritto, Dimitris Galanis, Yang Gao, Stefan Grünewald, Xianpei Han, Jialong Han, Michael A. Hedderich, Leonhard Hennig, Tom Hope, Ales Horak, Kasra Hosseini, Takumi Ito, Peter Jansen, Sai Jayanthi, Haiyun Jiang, Sudipta Kar, Mamoru Komachi, Lukas Lange, John Lee, Jing Li, Marina Litvak, Changsong Liu, Zhengzhong Liu, Wei Lü, Wolfgang Maier, Benjamin Marie, Στέλλα Μαρκαντωνάτου, Ivan Vladimir, Meza Ruiz, John R. Morris, Hamdy Mubarak, Philippe Muller, Preslav Nakov, Diane Napolitano, Pierre Nugues, Y. Oda, Tsuyoshi Okita, Daniel Ortega, Sebastian Padó, Sean Papay, Oren Pereg, Stelios Piperidis, Christopher Potts, F. Z. Qi, Carlos Ramisch, German Rigau, Pablo Ruiz Fabo, Irene Russo, Safi Samghabadi, Saurav Sahay, Sebastin Santy, David Schlangen, D Fabian, Hendrik Schmidt, Sanuj Schuff, Liang-Hsin Sharma, Michal Shen, Sameer Shmueli-Scheuer, Amy Singh, Konstantinos Siu, Mohammad Skianis, Yuanfeng Sohrab, Michael Song, Duyu Stewart, Juan-Manuel Tang, Natalia Torres-Moreno, Andrea Vanetik, Ivan Varga, Yan Vulić, Longyue Wang, Changhan Wang, Rui Wang, Leo Wang, Moritz Wanner, D. Wolf, Qiongkai Xiong, Tae Xu, Seid Yano, Dian Yimam, Huawei Noah's Ark Lab, Hong Kong, Adrián Pastor, López Monroy, Ahmed Abdelali, Amy Siu, Andréa Varga, Anoop Kunchukuttan, Arushi Raghuvanshi, Barbora Vidová Hladká, Ben Hachey, Carlos Escolano, Carolin Lawrence, Christos Christodoulopoulos, Clare Llewellyn, Dezhao Song, Dian Yu, Doug Downey, Eugene Kharitonov, Falavigna Schmidt, Fernando Daniele, Georgeta Alva-Manchego, German Bordea, Gianni Rigau, Guangyou Barlacchi, Hai Zhou, Hao Leong Chieu, Imed Li, Irene Zitouni, Meza Vladimir, Ivan Ruiz, James Vulić, J. Fan, John Kanerva, Jonas Lee, J. David Pfeiffer, Juan-Manuel Dexter, Jun Torres-Moreno, Jun Araki, Koji Zhao, Konstantinos Mineshima, Leo Skianis, Leonhard Wanner, Liang-Chih Hennig, Liang-Hsin Yu, Mamoru Shen, Margot Komachi, Marie-Jean Mieskes, Ma- Rina Meurs, Masoud Litvak, Michael Rouhizadeh, M Stewart, Ming Shmueli-Scheuer, M. Gong, Montse Yu, Natalia Cuadros, Nicolas Vanetik, Safi Niloofar, Oren Samghabadi, Pablo Pereg, Pascual Fabo, Patrick Martínez-Gómez, Petya Ernst, Pierre Osen- Ova, Pradipto Nugues, Preslav Das, Prokopis Nakov, Qiongkai Prokopidis, Rui Xu, Sameer Wang, S. Singh, Saurav Sharma, Sebastin Sahay, Seid Santy, Shajith Yimam, Siddharth Ikbal, Stelios Patwardhan, Sudipta Markantonatou, Sunayana Kar, Deyi Xiong, John Arevalo, Qun LiuOpenalex Unknown 2021 · DOI · OpenAlex

The EMNLP 2021 System Demonstrations represent a critical juncture in the evolution of Natural Language Processing (NLP). While research often focuses on incremental gains in model performance, the 42 systems presented in this volume address the "deployment gap"—the chasm between a high-performing research model and a usable, production-ready tool. This collection highlights a field-wide shift toward accessibility, interpretability, and robust developer infrastructure.

Standardizing the Ecosystem: Infrastructure and Libraries

A primary theme of the 2021 demonstrations is the "plumbing" of NLP. The standout contribution is the Hugging Face datasets library, which has since become the industry standard. By providing a unified interface to thousands of benchmarks, it solved a major friction point in the research pipeline. Other infrastructure tools like Box Embeddings introduced specialized geometric structures for representation learning, while fairseq S^2 advanced the state of speech synthesis. These systems demonstrate that the community values shared, stable infrastructure as much as architectural innovation.

Transparency and Sustainability: Interpretability and Green AI

As models grow in complexity, the need for interpretability becomes paramount. Tools like Thermostat and LMdiff provide visual interfaces that help researchers and developers understand why a model makes specific predictions, moving away from the "black box" paradigm.

Parallel to transparency is the rise of "Green AI." The inclusion of IrEne-viz, a tool for calculating and visualizing the energy consumption of Transformer models, signals a growing institutional concern for the environmental impact of large-scale AI. Efficiency is no longer just about inference speed; it is about the carbon footprint of the training and deployment cycle.

Democratizing NLP: Tools for Non-Experts

The 2021 proceedings also showcase a trend toward "low-code" or "no-code" NLP. Systems like SummerTime (for summarization) and TranslateLocally (for local CPU-based translation) are designed for non-experts, such as social scientists or journalists. This democratization allows researchers in other fields to leverage advanced NLP models without deep expertise in Python or machine learning.

Furthermore, human-in-the-loop annotation systems like LightTag, LexiClean, and UMR-Writer focus on the data labeling bottleneck. By making the annotation process faster and more consistent, these tools lower the barrier to entry for creating high-quality, domain-specific datasets.

Specialized and Real-Time Applications

The volume includes several high-stakes, domain-specific systems. ExcavatorCovid was designed to extract causal relations from COVID-19 literature, while the Press Freedom Monitor utilizes Twitter data to detect media violations. On the technical side, systems like MiSS (simultaneous translation) highlight the move toward real-time, edge-computing applications that prioritize privacy and low latency.

Conclusion and Future Directions

With a competitive acceptance rate of 32% (42 systems selected from 130 submissions), the EMNLP 2021 System Demonstrations reflect a mature and rigorous track. However, challenges remain. As noted by Co-Chairs Heike Adel and Shuming Shi, academic software often suffers from "bit rot" and scaling issues. Future work must focus on maintaining these tools beyond the initial publication and expanding their capabilities to handle multilingual data and enterprise-level workloads. Ultimately, these demonstrations serve as the bridge that carries NLP research from the laboratory into the real world.

Summary updated

BibTeX
@inproceedings{adel2021proceedings,
  author = {Heike Adel and Alan Akbik and Zeynep Akkalyoncu and Akari Asai and Eleftherios Avramidis and Wei Bi and Georgeta Bordea and Ari Bornstein and Aljoscha Burchardt and Eric Charton and Hai Leong Chieu and Simone Conia and D Croce and Montse Cuadros and Marina Danilevsky and Pradipto Das and Victor Dibia and Chenchen Ding and Carsten Eickhoff and Kshitij Fadnis and Nicolas Rodolfo Fauceglia and Anthony Ferritto and Dimitris Galanis and Yang Gao and Stefan Grünewald and Xianpei Han and Jialong Han and Michael A. Hedderich and Leonhard Hennig and Tom Hope and Ales Horak and Kasra Hosseini and Takumi Ito and Peter Jansen and Sai Jayanthi and Haiyun Jiang and Sudipta Kar and Mamoru Komachi and Lukas Lange and John Lee and Jing Li and Marina Litvak and Changsong Liu and Zhengzhong Liu and Wei Lü and Wolfgang Maier and Benjamin Marie and Στέλλα Μαρκαντωνάτου and Ivan Vladimir and Meza Ruiz and John R. Morris and Hamdy Mubarak and Philippe Muller and Preslav Nakov and Diane Napolitano and Pierre Nugues and Y. Oda and Tsuyoshi Okita and Daniel Ortega and Sebastian Padó and Sean Papay and Oren Pereg and Stelios Piperidis and Christopher Potts and F. Z. Qi and Carlos Ramisch and German Rigau and Pablo Ruiz Fabo and Irene Russo and Safi Samghabadi and Saurav Sahay and Sebastin Santy and David Schlangen and D Fabian and Hendrik Schmidt and Sanuj Schuff and Liang-Hsin Sharma and Michal Shen and Sameer Shmueli-Scheuer and Amy Singh and Konstantinos Siu and Mohammad Skianis and Yuanfeng Sohrab and Michael Song and Duyu Stewart and Juan-Manuel Tang and Natalia Torres-Moreno and Andrea Vanetik and Ivan Varga and Yan Vulić and Longyue Wang and Changhan Wang and Rui Wang and Leo Wang and Moritz Wanner and D. Wolf and Qiongkai Xiong and Tae Xu and Seid Yano and Dian Yimam and Huawei Noah's Ark Lab and Hong Kong and Adrián Pastor and López Monroy and Ahmed Abdelali and Amy Siu and Andréa Varga and Anoop Kunchukuttan and Arushi Raghuvanshi and Barbora Vidová Hladká and Ben Hachey and Carlos Escolano and Carolin Lawrence and Chang- Han Wang and C. Y. Liu and Christos Christodoulopoulos and Clare Llewellyn and Dezhao Song and Dian Yu and Doug Downey and Eugene Kharitonov and Falavigna Schmidt and Fernando Daniele and Georgeta Alva-Manchego and German Bordea and Gianni Rigau and Guangyou Barlacchi and Hai Zhou and Hao Leong Chieu and Imed Li and Irene Zitouni and I Russo and Meza Vladimir and Ivan Ruiz and James Vulić and J. Fan and John Kanerva and Jonas Lee and J. David Pfeiffer and Juan-Manuel Dexter and Jun Torres-Moreno and Jun Araki and Koji Zhao and Konstantinos Mineshima and Leo Skianis and Leonhard Wanner and Liang-Chih Hennig and Liang-Hsin Yu and Mamoru Shen and Margot Komachi and Marie-Jean Mieskes and Ma- Rina Meurs and Masoud Litvak and Michael Rouhizadeh and M Stewart and Ming Shmueli-Scheuer and M. Gong and Montse Yu and Natalia Cuadros and Nicolas Vanetik and Safi Niloofar and Oren Samghabadi and Pablo Pereg and Pascual Fabo and Patrick Martínez-Gómez and Petya Ernst and Pierre Osen- Ova and Pradipto Nugues and Preslav Das and Prokopis Nakov and Qiongkai Prokopidis and Rui Xu and Sameer Wang and S. Singh and Saurav Sharma and Sebastin Sahay and Seid Santy and Shajith Yimam and Siddharth Ikbal and Stelios Patwardhan and S Piperidis and Sudipta Markantonatou and Sunayana Kar and Deyi Xiong and John Arevalo and Qun Liu},
  title = {{Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations}},
  booktitle = {Openalex Unknown},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-demo},
}
Figure