TL;DR
This paper argues that algorithmic oppression in speech and language technologies (SLTs) stems from power imbalances embedded in datasets. It proposes a reflective framework, drawing on Data Feminism and Design Justice, to critically examine dataset content and documentation, focusing on identifying and addressing "data gaps" and their consequences for marginalized communities.
What Problem Does This Paper Address?
Algorithmic systems, including SLTs, can perpetuate and even amplify existing societal oppressions, a phenomenon termed "algorithmic oppression." This harm disproportionately affects marginalized communities. A key driver of this problem is the inherent bias and power dynamics within the datasets used to train and evaluate these technologies. While efforts exist to document datasets (e.g., Gebru et al., Bender & Friedman), this paper highlights the crucial need to go beyond what's included and actively investigate what's missing from datasets and their documentation. These "data gaps" reveal underlying power structures and can lead to exclusion and harm for specific groups. The paper aims to foster a critical conversation about power, representation, and bias in SLTs by providing a structured approach to analyzing these gaps.
Methods
The paper doesn't introduce new algorithms or quantitative methods. Instead, it proposes a qualitative, reflective framework for analyzing SLT datasets. This framework is built upon:
- Data Feminism Principles: Specifically, the seven principles outlined by D'Ignazio and Klein (2020): "examine power," "challenge power," "elevate emotion and embodiment," "rethink binaries and hierarchies," "embrace pluralism," "consider context," and "make labor visible."
- Design Justice Principles: Focusing on empowering communities, centering the voices of those impacted by design, and working towards sustainable, community-controlled designs.
- Sociolinguistics and Dataset Documentation: Leveraging existing work on how language variation is tied to social identity and the importance of comprehensive dataset documentation (e.g., Bender & Friedman's "data statements").
The core methodology involves a three-step reflective process:
- Identify gaps: Recognize missing content and documentation and their potential consequences.
- Investigate causes: Understand why these gaps exist within the broader social, historical, and technical context.
- Imagine alternatives: Envision how datasets could be compiled and used differently to create more just and less harmful technologies.
Datasets
The paper uses two datasets as case studies to illustrate its framework, not for quantitative evaluation:
- Common Voice English (release 7.0): A crowd-sourced speech dataset compiled by Mozilla with the aim of creating diverse speech datasets for ASR development.
- Switchboard-2 (SWB-2): A collection of telephone conversations collected by the Linguistic Data Consortium (LDC) for speaker recognition systems, later used for ASR benchmarks.
The paper notes that these datasets were compiled in different ways by different institutions, resulting in distinct data gaps.
Key Insights and Ideas
- "Data gaps" are as important as "data statements": While documenting what's in a dataset is crucial, understanding what's missing is equally vital for revealing power imbalances and potential harms.
- Algorithmic oppression is a reproduction of societal oppression: Biases in SLTs are not just technical "bugs" but predictable outcomes of existing structural inequalities reflected in data.
- Inclusion can be harmful: Simply including more data from marginalized groups doesn't guarantee equity; it can sometimes expose them to further marginalization or surveillance if not handled critically.
- Datasets are political tools: They are not neutral but are shaped by social relations and power dynamics, and their compilation involves deliberate choices about who and what is included or excluded.
- Context is paramount: Evaluating datasets and their gaps requires understanding their specific compilation context and intended (or actual) deployment context.
- Refusal as a strategy: In some cases, the most effective way to challenge power is to refuse to build a technology or compile a dataset.
- Embodiment matters: The physical presence and experiences of speakers, annotators, and curators are integral to understanding data and its potential harms, countering the "disembodied" framing often applied to ML.
Key Results
This paper is primarily a conceptual and methodological contribution, not one that presents quantitative results from experiments. Its "results" are the insights derived from applying its reflective framework to the chosen datasets. The key findings are qualitative and revolve around the identification and analysis of data gaps and their implications:
- Common Voice English: Gaps identified include a lack of accent labels for a significant portion of recordings, underrepresentation of female and younger speakers, and the absence of many global Englishes from Africa and Asia.
- Switchboard-2: Gaps include the absence of recorded race or ethnicity, underrepresentation of older speakers, and the likely skew towards native US English speakers due to recruitment methods. The paper also notes that while topics were chosen to spark discussion, some controversial topics could elicit harmful discourses.
The paper demonstrates how these gaps can constrain appropriate use cases and make it difficult to evaluate predictive bias, particularly concerning race.
Limitations and Future Work
- Limitations: The paper acknowledges that addressing data gaps can be difficult and that its proposed questions are not exhaustive. It also notes that while the datasets themselves might not directly cause harm, their use in specific contexts can lead to adverse outcomes. The paper also touches upon the potential for "ornamental intersectionality" where concepts like diversity and inclusion are used symbolically without addressing underlying structural inequalities.
- Future Work: The paper implicitly suggests future work in several areas:
- Developing more inclusive data compilation and annotation practices that directly involve marginalized communities.
- Creating "counterdata" to fill deliberate gaps.
- Further research into how to ethically and effectively fill documentation gaps, especially regarding sensitive demographic information.
- Encouraging a broader conversation and critical engagement with the political nature of SLT datasets.
- Developing more robust methods for evaluating the impact of data gaps on SLT performance across diverse user groups.
Summary updated