Export
Jennifer Hsia
· 1 previous affiliation
- Carnegie Mellon UniversityCurrent
- Current
- PhD student · 2021–2026
Research Overview
Jennifer Hsia focuses on the systematic evaluation and optimization of Retrieval-Augmented Generation (RAG) systems, with a particular emphasis on the interaction between retrieval quality and reader model robustness. Her work addresses the critical challenge of Filtering Noise in Knowledge-Augmented Language Models, investigating how large language models (LLMs) process external context and why performance often degrades when models are exposed to irrelevant or noisy information. By developing the RAGGED framework, Hsia provides a methodology for analyzing the design space of RAG architectures, moving beyond simple accuracy metrics to explore the underlying factors that determine a system's stability and scalability in real-world applications.
Research Trajectory
Hsia’s research trajectory is characterized by a transition from broad architectural benchmarking to a focused investigation of noise sensitivity and system reliability. In 2024, her work established a foundational understanding of the RAG design space. Through the initial development of the RAGGED framework, Hsia examined how various configurations—including the choice of retriever and reader models, context length, and context quality—impacted performance across diverse task types. This period of research was instrumental in identifying that improvements in retrieval do not always translate into proportional gains in the final output, a phenomenon largely driven by the "reader" model's inability to filter out non-pertinent data.
In 2025, Hsia’s focus shifted toward the longitudinal and structural properties of these systems, specifically addressing Performance Plateaus and Noise Sensitivity in Retrieval-Augmented Generation. This phase of her work introduced more rigorous metrics for stability and scalability, challenging the prevailing assumption that increasing the volume of retrieved documents necessarily improves model grounding. Her recent analysis suggests that as the "signal-to-noise" ratio decreases within an expanded context window, the distraction caused by irrelevant documents can outweigh the benefits of including the "gold" evidence. This trajectory reflects a deepening interest in the "distraction phenomenon" and the necessity of Reducing Contextual Noise for Robust Reasoning.
Current Directions
Hsia’s current research is centered on the informed design of RAG systems that can maintain high performance in the presence of "hard negatives"—passages that are semantically related to a query but factually irrelevant. This work aligns with the broader academic interest in Distractor-Augmented Training for Robust Retrieval-Augmented Generation. Her recent findings emphasize that the success of a RAG system depends less on the absolute quality of the retriever and more on the reader model's capacity for selective integration.
A primary focus of her current work is the refinement of the RAGGED framework to serve as a diagnostic tool for identifying "noise-sensitive" readers. By isolating the reader's performance from the retriever's accuracy, Hsia aims to provide developers with the means to select models that are inherently more robust to the clutter often found in large-scale search results. This research is particularly relevant as context windows in LLMs continue to expand, making the effective processing of long-form, noisy input a central problem in natural language processing.
Representative Work
Hsia’s contributions are primarily concentrated in the development and application of the RAGGED evaluation framework, as detailed in the following representative publications:
In RAGGED: Towards Informed Design of Retrieval Augmented Generation Systems (2024), Hsia and her colleagues introduced a framework to investigate the complex dependencies between RAG components. As the lead author, Hsia conducted an extensive study on how context length and quality affect different reader models. The study revealed a significant disparity in how models handle irrelevant information: while some readers showed consistent benefits from additional context, others experienced performance degradation when exposed to noise. This work highlighted the necessity of tuning RAG systems based on the specific sensitivity of the reader model rather than focusing solely on retriever recall.
A subsequent iteration of this work, RAGGED: Towards Informed Design of Retrieval Augmented Generation Systems (2025), further formalized these findings for the broader research community. This research demonstrated that the integration of external context is not a plug-and-play solution but requires careful configuration of the reader-retriever pair. Hsia’s analysis showed that retriever improvements often hit a point of diminishing returns if the reader model is not equipped to handle the increased volume of potentially distracting information.
In RAGGED: Towards Informed Design of Scalable and Stable RAG Systems (2025), Hsia extended the framework to evaluate the scalability and stability of RAG architectures. This paper introduced a "hard truth" regarding modern RAG systems: more data is not always better. By measuring how systems behave as the number of retrieved documents increases, Hsia demonstrated that a system's stability is heavily dependent on the reader's ability to ignore irrelevant noise. This work provides a critical metric for developers, suggesting that the most "scalable" systems are those that maintain performance even as the signal-to-noise ratio drops.
The evidence currently available for Hsia’s research is concentrated within the 2024-2025 period and focuses heavily on the RAGGED framework. While this provides a clear and deep view of her contributions to RAG evaluation, her work in other areas of machine learning or prior to this period is not documented in the current corpus.
Profile updated
Loading new papers…
Other researchers whose papers cluster into the same research directions — a proxy for overlapping technical interests and shared intellectual neighborhoods.
See something wrong? Report a mistake