Export
Ani Nenkova
· 1 previous affiliation
- Adobe Systems (United States)Since 2021
- Principal Scientist · Since 2021
Research Overview
Ani Nenkova focuses on natural language processing, text summarization, and the evaluation of generative models. Her early research explored discourse structure, local coherence, and methods for assessing summary content without gold-standard references. More recently, her work addresses the robustness of large language models, out-of-distribution detection, and the standardized measurement of text diversity and factual consistency.
Research Trajectory
Ani Nenkova's early research established methodologies for evaluating text summarization. In Evaluating Content Selection in Summarization: The Pyramid Method (2004), she addressed the area of The Subjectivity Problem in Text Distillation Evaluation and the broader direction of Decoupling Content and Fluency in Text Generation Evaluation. This work formalized information importance as a distribution of human consensus, providing a structured approach to multi-reference evaluation for natural language generation.
Her focus subsequently expanded to discourse structure and coherence, contributing to the area of Recognizing Implicit Discourse Relations. With collaborators, she published Using Syntax to Disambiguate Explicit Discourse Connectives in Text (2009), which demonstrated that syntactic features significantly improve the identification of discourse versus non-discourse usage, advancing work in the direction of Disambiguating Functional and Semantic Roles of Discourse Markers. The following year, Creating Local Coherence: An Empirical Assessment investigated the interaction between discourse relations and entity transitions, modeling text as hierarchical discourse trees to better understand local coherence in news articles.
Continuing her work on summarization, she explored reference-free evaluation techniques in Automatically Assessing Machine Summary Content Without a Gold Standard (2013). In 2014, her research group analyzed how discourse structures impact cross-lingual tasks in Assessing the Discourse Factors that Influence the Quality of Machine Translation. This study revealed that mismatches in sentence boundaries and the use of explicit connectives cause significant degradation in translation quality between Chinese, Arabic, and English. In 2015, System Combination for Multi-document Summarization presented a supervised framework that generates candidate summaries by combining sentences from multiple systems, leveraging cross-document consensus to identify salient content.
Nenkova's work also addressed the practical challenges of dataset creation and model adaptation. She oversaw research on Predicting Annotation Difficulty to Improve Task Routing and Model Performance for Biomedical Information Extraction (2019), which sought to optimize the annotation process for specialized domains. As language models became more prevalent, she investigated their robustness over time in Temporal Effects on Pre-trained Models for Language Processing Tasks (2021). This research highlighted how models like BERT and RoBERTa handle semantic shift, showing that retraining on recent data—even if noisy—yields significant performance gains.
In recent years, her trajectory has heavily emphasized the evaluation, robustness, and deployment safety of generative models. Factual or Contextual? Disentangling Error Types in Entity Description Generation (2023) introduced a framework to separate nonfactual errors from incongruous ones, identifying a "familiarity gap" where models generate plausible but false information for obscure entities. She also contributed to Context-aware Information-theoretic Causal De-biasing for Interactive Sequence Labeling (2023), addressing context and feedback biases in privacy-preserving, interactive learning setups. In Summarization from Leaderboards to Practice: Choosing A Representation Backbone and Ensuring Robustness (2023), her team demonstrated that training models like BART on diverse, mixed-domain datasets produces more robust summarization systems than relying on single massive datasets. Additionally, A Critical Analysis of Document Out-of-Distribution Detection (2023) provided an in-depth study of how multi-modal information in documents can be leveraged to trigger conservative fallback policies during deployment.
Most recently, she participated in developing tools for the standardized measurement of text diversity in Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scores (2024), showing that length-controlled compression ratios can effectively replace computationally expensive metrics. Finally, How Much Annotation is Needed to Compare Summarization Models? (2024) demonstrated that preferences between summarization models can stabilize with as few as 50 test samples, significantly reducing the cost of evaluating large language models.
Current Directions
The most recent papers in the evidence address the efficient evaluation and robustness of large language models. Recent work investigates how to streamline the assessment of generative models, demonstrating that reliable model comparisons can be achieved with significantly fewer annotated samples than traditionally required. Another major focus is the standardization of text diversity measurements, utilizing length-controlled compression ratios to evaluate model outputs efficiently. Additionally, current research explores the deployment safety of document understanding models through out-of-distribution detection and seeks to disentangle different types of hallucinations—specifically distinguishing between contextually plausible but nonfactual errors and factually true but irrelevant generations.
Profile updated
Loading new papers…
Other researchers whose papers cluster into the same research directions — a proxy for overlapping technical interests and shared intellectual neighborhoods.
See something wrong? Report a mistake