Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning

Timnit GebruKDD 2020 · DOI

TL;DR

Machine learning's fairness, accountability, and transparency issues often stem from data collection practices. This paper argues for a new ML specialization focused on data collection methodologies, drawing inspiration from the long-standing practices of archives and libraries. By adapting archival strategies for consent, inclusivity, power, transparency, and ethics, ML can build more responsible and equitable systems.

What Problem Does This Paper Address?

The paper tackles a fundamental yet often overlooked problem in machine learning (ML): the data collection and annotation process. The authors argue that many issues in ML fairness, accountability, and transparency are rooted in how data is gathered and prepared. This is particularly true for "sociocultural data," which can easily perpetuate societal biases and harm vulnerable groups if collected haphazardly. The paper highlights that while ML research often focuses on model development, the critical upstream process of data collection remains largely unsystematic and lacks established best practices, leading to what is termed the "wild west" of data collection.

Methods

The paper's core methodological contribution is drawing parallels between archival science and ML data collection. It proposes that ML researchers can learn from the institutional frameworks, procedures, and language developed over centuries by archives and libraries to address challenges related to data collection. The authors analyze archival approaches across five key areas: consent, inclusivity, power, transparency, and ethics & privacy. They then map these archival strategies onto potential applications within ML, advocating for a more "interventionist" approach to data collection rather than a "laissez-faire" one. This involves actively shaping data collection processes with specific goals and ethical considerations in mind, rather than simply scraping available data.

Datasets

The paper does not introduce new datasets for training or evaluation. Instead, it uses existing ML datasets and practices as examples to illustrate the problems and proposes how archival methods could improve their collection. The case study discusses OpenAI's WebText dataset, used for training the GPT-2 language model, as an example of a "laissez-faire" approach that could benefit from more interventionist strategies.

Key Insights and Ideas

  • Archives as a Model for Responsible Data Collection: Archives, with their long history of collecting and preserving human information, have developed robust methodologies for handling sensitive data, ensuring representation, and maintaining transparency. These practices offer a valuable blueprint for ML.
  • The "Wild West" of ML Data Collection: Current ML data collection is often indiscriminate, driven by availability and efficiency rather than a systematic, ethical framework, leading to the perpetuation of societal biases.
  • The Need for an "Interventionist" Approach: Instead of passively collecting data, ML practitioners should actively intervene in the data collection process to mitigate biases, ensure inclusivity, and uphold ethical standards. This involves defining clear objectives and collection policies.
  • Five Pillars of Archival Wisdom for ML: The paper identifies five key areas where archives excel and ML can learn:
  • Inclusivity: Archives use Mission Statements and Collection Policies to actively seek out underrepresented groups and ensure diverse representation.
  • Consent: Community and participatory archives emphasize open-ended input and allow communities to define their own categorization and representation.
  • Power: Data consortia in archives allow for resource sharing and collective storage, mitigating power imbalances between institutions.
  • Transparency: Rigorous appraisal processes and detailed record-keeping in archives provide a transparent history of data collection decisions.
  • Ethics & Privacy: Archives have established codes of ethics and conduct, often enforced through professional organizations and committees.
  • The Value of Full-Time Data Collectors: Promoting full-time roles for data collection and management, akin to archivists, can incentivize adherence to ethical standards and professional conduct.

Key Results

The paper does not present quantitative results in the traditional sense, as it is not an empirical study of ML model performance. Instead, its key "results" are conceptual and prescriptive:

  • A Framework for Improvement: The paper provides a structured framework for improving ML data collection by adapting archival practices.
  • Identification of Gaps: It clearly articulates the significant gaps in current ML data collection practices compared to established archival methods.
  • Call for a New Specialization: The authors advocate for the formation of a new specialization within ML focused on data collection and annotation methodologies.

Limitations and Future Work

The authors acknowledge several limitations:

  • Scalability and Cost: Implementing archival-inspired methods, such as hiring full-time staff, maintaining detailed documentation, and establishing consortia, can incur significant overhead, time, and financial costs, especially for large-scale ML datasets.
  • Motivation Differences: ML datasets are often commercially motivated and tied to profit or defense objectives, unlike the preservation-focused mission of archives. This difference in motivation can create challenges in adopting archival practices, particularly regarding data proprietary concerns.
  • Power Dynamics in Selection: While interventionist models aim to diffuse power, highly selective data collection can still concentrate power in the hands of a few individuals or institutions, raising concerns about undue social or political influence.

Future work suggested includes:

  • Developing Data Consortia: Encouraging the formation of data consortia in ML to share resources and reduce costs.
  • Establishing Professional Organizations: Creating professional organizations for ML data collectors that work by membership to enforce ethical guidelines.
  • Supporting Community Archives: Promoting and supporting community archives as a model for participatory data collection.
  • Forming a Dedicated Subfield: Advocating for a subfield dedicated to the data collection and annotation process in ML.
  • Further Research on Incentive Models: Investigating incentive models that encourage ethical data collection practices in ML.
  • Drawing from Other Disciplines: Exploring lessons from other social sciences like sociology, psychology, history, and anthropology for dealing with direct human subjects and cultural sensitivities.

Summary updated

BibTeX
@inproceedings{gebru2020lessons,
  author = {Timnit Gebru},
  title = {{Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2020},
  doi = {10.1145/3394486.3409559},
}
Figure