Understanding the Dataset Practitioners Behind Large Language Models

Crystal Qian, Emily Reif, Minsuk Kahng

This paper, recently published by researchers at Google, takes a human-centric look at the "data work" that powers modern Large Language Models (LLMs). While the industry often focuses on model architectures and parameter counts, this study shines a light on the people—the dataset practitioners—responsible for the messy, subjective, and critical task of curating and evaluating the data that goes into (and comes out of) these models.

TL;DR

The paper defines the emerging role of the "dataset practitioner" and reveals a striking paradox: while data quality is the undisputed top priority in LLM development, there is no consensus on what "quality" actually means. Consequently, practitioners rely on manual "eyeballing" in spreadsheets and custom Python scripts, often bypassing sophisticated interpretability tools in favor of their own intuition.

What Problem Does This Paper Address?

As we move from the "big data" era (where volume was king) to the LLM era, the focus has shifted toward high-quality, curated datasets. However, LLM data is largely unstructured text, making traditional statistical analysis insufficient.

The authors identify a gap in our understanding of the people doing this work. Who are they? What are their workflows? And why, despite the proliferation of AI explainability (XAI) and visualization tools, do many practitioners still feel like they are "flying blind"? The paper seeks to define this role and identify the bottlenecks preventing the adoption of more rigorous data evaluation standards.

Methods

The researchers employed a two-phase qualitative approach:

  1. Retrospective Analysis: They analyzed the organizational structure and internal projects at Google to define the "dataset practitioner" role. They identified representative tasks ranging from curating benchmarks to making "go/no-go" deployment decisions.
  2. Semi-Structured Interviews: They conducted 30-minute interviews with 10 practitioners (N=10) across three domains: Tooling (building the infra), Modeling (training the models), and Evaluation (testing for safety and performance).
  3. Thematic Analysis: The interviews were transcribed and coded to identify recurring themes, challenges, and tool usage patterns.

Datasets

This is a qualitative human-computer interaction (HCI) study, so it does not introduce a new machine learning dataset. Instead, it categorizes the types of datasets practitioners interact with:

  • Training Sets: Pre-training corpora (web scrapes), Supervised Fine-Tuning (SFT), and Reinforcement Learning from Human Feedback (RLHF) data.
  • Evaluation Sets: Safety benchmarks, model outputs (for side-by-side comparison), and conversational data from "in-the-wild" user interactions.

Key Insights and Ideas

  • The "Eyeball" Standard: Every single participant reported that their primary method for validating data quality was visually scanning examples in a spreadsheet. As one participant put it: "I eyeball data... It’s all my own intuition."
  • The Subjectivity of Quality: There is no "North Star" for data quality. What one team considers "high quality" for a chatbot's tone might be seen as "noisy" or "unhelpful" by another. This lack of consensus makes it nearly impossible to build "one-size-fits-all" evaluation tools.
  • The Tooling Gap: Practitioners are caught in a "chicken and egg" problem. They don't use specialized tools because the tools don't fit their custom needs, but tools don't evolve because practitioners are off writing custom, one-off Python notebooks to solve immediate problems.
  • Confirmation Bias is a Known Risk: Practitioners admitted that "eyeballing" data leads to confirmation bias—they often only look for what they expect to see, or only check the data when the model's output looks "wrong."
  • The Spreadsheet as a "Social" Tool: Spreadsheets remain dominant not just because they are easy to use, but because they are easy to share with non-technical stakeholders (like legal or policy teams) who need to sign off on safety datasets.

Key Results

The study produced a Taxonomy of the Dataset Practitioner, mapping out their processes, tools, and challenges (summarized in Table 2 of the paper).

  • Consensus on Challenges: 100% of participants cited "Assessing Data Quality" as a major challenge.
  • Custom Code is King: 7 out of 9 participants mentioned writing custom Python code in notebooks (like Colab) to explore data, citing the need for flexibility that current UI-based tools don't provide.
  • Desired Features: When asked what they actually wanted in a tool, practitioners converged on three things:
  • Better summarization of data slices.
  • Automated safety/toxicity thresholding.
  • Better ways to track numeric distributions (like token length).

Limitations and Future Work

The primary limitation is the small, localized sample size: 10 practitioners, all from a single large technology company (Google). Their workflows might not represent those at smaller startups or in academia.

Future Work suggested by the authors includes:

  • Developing conceptual frameworks to help teams reach a consensus on what "quality" means in specific contexts.
  • Standardizing metrics for features like data diversity and toxicity.
  • Designing "flexible" tooling that combines the ease of a spreadsheet with the power of custom Python analysis to reduce the high effort currently required for rigorous data auditing.

Summary updated

Abstract

As large language models (LLMs) become more advanced and impactful, it is increasingly important to scrutinize the data that they rely upon and produce. What is it to be a dataset practitioner doing this work? We approach this in two parts: first, we define the role of "dataset practitioners" by performing a retrospective analysis on the responsibilities of teams contributing to LLM development at a technology company, Google. Then, we conduct semi-structured interviews with a cross-section of these practitioners (N=10). We find that although data quality is a top priority, there is little consensus around what data quality is and how to evaluate it. Consequently, practitioners either rely on their own intuition or write custom code to evaluate their data. We discuss potential reasons for this phenomenon and opportunities for alignment.
BibTeX
@inproceedings{qian2024understanding,
  author = {Crystal Qian and Emily Reif and Minsuk Kahng},
  title = {{Understanding the Dataset Practitioners Behind Large Language Models}},
  booktitle = {CHI Extended Abstracts},
  year = {2024},
  doi = {10.1145/3613905.3651007},
}

Loading related content...

Figure