TL;DR
This paper surveys how benchmark datasets are shared on popular platforms, revealing a critical lack of documentation regarding social impact and limitations. The authors argue for a more comprehensive approach, advocating for the inclusion of "social impact metadata" and the establishment of data curators to manage this information, ensuring safer and more responsible use of these datasets.
What Problem Does This Paper Address?
In the rapidly evolving fields of Machine Learning (ML) and Natural Language Processing (NLP), benchmark datasets are crucial for evaluating model performance and driving progress. However, recent research has begun to question the validity and potential harms associated with these datasets, highlighting issues like generalizability, documentation practices, and social impact. This paper focuses on the latter: the way benchmark datasets are shared and documented on platforms. The authors observe that while platforms excel at providing technical details for easy dataset usage, they often fall short in offering comprehensive information about potential social impacts, biases, and limitations. This "data friction" can lead to datasets being misinterpreted or misused, potentially causing societal harm. The paper aims to bridge this gap by examining current sharing practices and proposing a more robust framework for documenting and managing benchmark datasets.
Methods
The authors conducted a survey of four prominent benchmark dataset sharing platforms: HuggingFace, PaperswithCode, TensorFlow, and PyTorch. Their methodology involved analyzing how these platforms present datasets, what metadata is shared, and what is omitted. They specifically looked for information related to dataset creation, limitations, and social impact. Drawing on concepts from Library and Information Science, particularly data curation, they propose a framework that extends beyond traditional descriptive and administrative metadata to include "social impact metadata." This involves identifying and documenting potential societal harms, biases, and limitations of a dataset.
Datasets
The paper does not introduce new datasets. Instead, it analyzes the documentation and sharing practices of existing benchmark datasets on platforms like HuggingFace, PaperswithCode, TensorFlow, and PyTorch. Examples of benchmark datasets mentioned include GLUE, SuperGLUE, SQuAD, Seteroset, and CrowS for NLP tasks, as well as the hatexplain and ethos datasets for specific bias detection.
Key Insights and Ideas
- Overemphasis on Usability, Underemphasis on Impact: Current platforms prioritize making datasets easy to access and use for ML/NLP tasks, often providing detailed instructions for loading and splitting data. However, they largely neglect to document crucial information about the dataset's limitations and potential social impacts.
- Lack of Consensus on Social Impact Metadata: There's no standardized understanding or consistent documentation of what constitutes "social impact metadata." This ambiguity makes it difficult for users to assess potential risks.
- The "Data Friction" Problem: Even with good documentation, the inherent complexity of data and its reuse can lead to misinterpretations or loss of information, a phenomenon the authors refer to as "data friction." This highlights the need for proactive and clear communication of potential harms.
- The Role of the Data Curator is Crucial: The authors propose that data curators, specialists within the sharing platforms, should be responsible for identifying, documenting, and organizing social impact metadata. This role extends beyond simply collecting and selecting data to critically assessing its potential societal implications.
- Beyond Descriptive Metadata: While descriptive (e.g., title, keywords) and administrative (e.g., version, license) metadata are common, they are insufficient. The paper argues for the necessity of "social impact metadata" to inform users about potential biases, ethical considerations, and limitations.
- Examples of Social Impact: The paper points to examples like the "Leéruda" chatbot incident, where AI systems can jeopardize marginalized groups, underscoring the real-world consequences of unaddressed social impacts in datasets.
Key Results
The survey revealed that while platforms like HuggingFace offer some sections for social impact, the information is often vague, inconsistent, or mixed with technical limitations. PaperswithCode, TensorFlow, and PyTorch primarily focus on descriptive and administrative metadata, with little to no discussion of social impacts. HuggingFace's "data cards" are the closest to addressing social impact, but even here, the content varies widely, ranging from positive impacts to potential risks, with no clear consensus. The paper highlights that the documentation of limitations is also inconsistent, with some datasets mentioning contextual limitations (e.g., language focus) and others technical ones (e.g., dataset size).
Limitations and Future Work
The authors acknowledge that the current discussion around the social impact of benchmark datasets is missing from sharing platforms. They note that while dataset creators play a crucial role, their focus is on this paper is on the platforms. A key limitation identified is the lack of a clear consensus on who is responsible for organizing social impact information.
Future work suggested by the paper includes:
- Developing Standardized Social Impact Metadata: Creating a clear taxonomy and guidelines for documenting social impact, including demographic statistics of the data and annotators, and potential harms.
- Formalizing the Role of the Data Curator: Establishing the data curator as a key figure responsible for assessing and communicating the social impact of datasets.
- Encouraging a Culture of Responsible Data Sharing: Promoting a broader understanding and adoption of social impact considerations within the ML/NLP community.
Summary updated