Large Image Datasets: A Pyrrhic Win for Computer Vision?

Abeba Birhane, Vinay Uday PrabhuWACV 2021

TL;DR

This paper critically examines the ethical implications of large-scale image datasets (LSVDs) in computer vision, arguing that their widespread use, exemplified by ImageNet, has come at a significant cost to individual privacy, consent, and the perpetuation of harmful stereotypes. The authors propose a quantitative auditing framework and advocate for mandatory Institutional Review Boards (IRBs) for LSVD curation to foster a more just and ethical AI landscape.

What Problem Does This Paper Address?

The paper tackles the growing concern surrounding the ethical shortcomings of large-scale image datasets (LSVDs) that underpin much of modern computer vision research. While these datasets have fueled remarkable progress, the authors argue that their creation and use often disregard fundamental principles of informed consent, privacy, and agency. They highlight issues ranging from the inclusion of non-consensual imagery (e.g., up-skirt photos) to the perpetuation of harmful stereotypes through biased labeling. The paper questions whether the immense benefits of these datasets are worth the societal and individual harms they can inflict, framing this as a "Pyrrhic win" for computer vision.

Methods

The authors employ a multi-pronged approach to audit the ImageNet dataset:

  • Quantitative Census: They perform a cross-sectional, model-based quantitative census of the ImageNet-ILSVRC-2012 dataset. This involves analyzing factors such as age, gender, NSFW content scoring, class-wise accuracy, human-cardinality analysis, and the semanticity of class labels.
  • Model-Based Analysis: They leverage pre-trained models like DEX and InsightFace for gender and age estimation, and a NSFW scoring model to identify problematic content.
  • Hybrid Human-in-the-Loop Approach: They combine automated analysis with manual curation. This includes segmenting the dataset using pre-trained models and then hand-labeling subsets to identify specific categories of concern, such as misogynistic images and classes containing infants.
  • Literature Review and Ethical Framework: The paper draws heavily on existing work in Science and Technology Studies (STS) and social sciences to contextualize the ethical challenges and propose solutions.

Datasets

  • ImageNet-ILSVRC-2012: This is the primary dataset analyzed in the paper. The authors focus on its subset used for the ILSVRC challenge.
  • Tiny Images Dataset: Mentioned as another LSVD with similar issues, particularly regarding offensive labels.
  • Other LSVDs: The paper references other large datasets like JFT-300M, Open Images, and Tencent-ML, often in comparison to ImageNet.

The authors also introduce curated meta-datasets derived from their audit, including CAG statistics, NSFW scores, accuracy metrics, and semanticity data, which are open-sourced for community use.

Key Insights and Ideas

  • The "Pyrrhic Win": The paper argues that the immense progress in computer vision driven by LSVDs like ImageNet has come at a significant ethical cost, eroding privacy and perpetuating harm.
  • The "WordNet Effect": The reliance on WordNet as a taxonomic backbone for datasets like ImageNet can embed and amplify existing societal biases and problematic classifications.
  • Creative Commons Fallacy: The interpretation of Creative Commons licenses as a blanket permission for data use in LSVDs is fundamentally flawed, as these licenses primarily address copyright, not privacy or consent.
  • "Blood Diamond Effect": Models trained on ethically compromised datasets can inherit and propagate these issues downstream, leading to biased or harmful applications.
  • The Need for Mandatory IRBs: The authors strongly advocate for the establishment of mandatory Institutional Review Boards (IRBs) for LSVD curation, similar to those in medical and psychological research, to ensure ethical oversight.
  • Dataset Audit Cards: They propose the dissemination of "dataset audit cards" (akin to model cards) to transparently document dataset goals, curation procedures, and known shortcomings.

Key Results

  • Identification of Problematic Content: The audit identified 62 "confirmed misogynistic images" across categories like "bikini," "maillot," and "brassiere," including instances of beach voyeurism and exposed private parts.
  • Presence of Infants: The dataset contains 30 classes with images of infants, raising concerns about consent and exploitation.
  • Scale of Data: The analysis revealed 83,436 images with persons, encompassing 101,070 to 132,201 individuals, representing 8-10% of the dataset.
  • Gender and Age Skewness: The analysis highlighted gender and age estimation biases when using automated tools on ImageNet classes. For example, the mean age for males was 33.24, while for females it was 25.58.
  • Open-Sourced Resources: The authors have open-sourced all code and curated meta-datasets to facilitate further research and auditing.

Limitations and Future Work

  • Limitations of Automated Tools: The authors acknowledge that the automated tools used for gender and age estimation have their own ethical limitations and can be prone to bias. They emphasize that gender classification based on appearance is scientifically flawed and carries a high risk of systemic abuse.
  • Complexity of Ethical Challenges: The paper notes that many of the ethical challenges are deeply rooted in social and cultural structures, and there is no single, straightforward solution.
  • Future Work: The authors suggest several paths forward, including:
  • Remove, Replace, and Open Strategy: Removing offensive labels and images, replacing them with ethically sourced alternatives, and making datasets open.
  • Differentially Private Obfuscation: Using techniques like DP-Blur to protect individual identities in images.
  • Synthetic Data: Exploring the use of synthetic images (e.g., GAN-generated or sketches) for training.
  • Ethics-Reinforced Filtering and IRBs: Implementing explicit ethical guidelines during curation and establishing IRBs.
  • Dataset Audit Cards: Promoting the use of audit cards for transparency.

Summary updated

BibTeX
@inproceedings{birhane2021large,
  author = {Abeba Birhane and Vinay Uday Prabhu},
  title = {{Large Image Datasets: A Pyrrhic Win for Computer Vision?}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
}
Figure