Auditing Saliency Cropping Algorithms

Abeba Birhane, Vinay Uday Prabhu, John WhaleyWACV 2022

TL;DR

This paper audits the saliency-based image cropping (SIC) algorithms used by Twitter, Google, and Apple, revealing significant racial and gender biases, as well as a concerning "male-gaze" phenomenon. The authors introduce a methodology and datasets to systematically evaluate these biases, highlighting the need for greater transparency and ethical considerations in AI-driven image manipulation.

What Problem Does This Paper Address?

Saliency-based image cropping (SIC) is a ubiquitous technology used by major platforms like Twitter, Google, and Apple to automatically crop user-uploaded images. The goal is to retain the "most interesting" parts of an image. However, recent controversies, particularly on Twitter, have raised concerns about potential biases in these algorithms, specifically regarding racial discrimination and a "male-gaze" effect where women are disproportionately cropped in ways that emphasize certain body parts. This paper aims to systematically audit and compare the SIC algorithms from these three major tech companies to understand the extent and nature of these biases, and to investigate whether they are inherent to the algorithms or influenced by the training data and cropping policies.

Methods

The authors conducted a comparative audit of the SIC frameworks from Twitter, Google (CROP_HINTS API), and Apple (Attention-Based Saliency Cropping - ABSC). Their methodology involved:

  • Male-Gaze Artifact Analysis: They curated a dataset of real-world, full-body images of women from red-carpet events, characterized by logo-littered backdrops and specific aspect ratios. These images were passed through each SIC framework, and the resulting crops were manually labeled for the presence of male-gaze-like artifacts.
  • Racial and Gender Bias Analysis: A synthetic dataset of 3x1 image grids was created, featuring combinations of Black and White individuals of Male and Female genders. This dataset was designed to mimic the format of a viral image that highlighted racial bias. The authors then analyzed the "survival ratios" of faces in the cropped images to quantify biases.
  • Comparative Framework Analysis: They meticulously compared the technical specifications and outputs of the three SIC frameworks, noting differences in saliency map generation, confidence scores, aspect ratio handling, and availability of documentation.

Datasets

  • Male-Gaze Dataset: A custom dataset of 336 real-world images was curated from public sources (specifically, the Walt Disney Television official Flickr account). These images featured full-body shots of women from red-carpet events with logo-littered backgrounds and specific height-to-width ratios (>= 1.25).
  • Synthetic Race-Gender Dataset: A dataset of 3000 random images was generated, with 500 images sampled from each of six race-gender configurations (BM, BF, WM, WF). These images were derived from the Chicago Faces Dataset (CFD), which provides controlled images of individuals who self-identified their race and gender, and were specifically selected for neutral expressions.

Key Insights and Ideas

  • Male-Gaze is Real and Prevalent: The study provides empirical evidence that the "male-gaze" cropping phenomenon is not just anecdotal but occurs with a worryingly high prevalence rate (~41%) in specific contexts (long, thin, full-body images of women with busy backgrounds).
  • Twitter's Framework Uniquely Prone to Male-Gaze: While all three frameworks exhibited biases, Twitter's saliency cropping framework was found to uniquely elicit a high prevalence of male-gaze cropping.
  • Racial and Gender Biases are Systemic: Beyond the male-gaze issue, all three audited frameworks demonstrated acute racial and gender biases in how they cropped faces, particularly in 3x1 grids.
  • "Fortuitous" Non-MGL Crops Exist: The study identified instances where non-male-gaze crops occurred due to serendipity, where a focal point on a background logo coincidentally resulted in a more balanced crop.
  • Algorithmic Brittleness and Ethics-Washing: The authors highlight the extreme fragility of these cropping algorithms, where minor changes in input data can drastically alter survival ratios, making them susceptible to "ethics-washing" to fit desired narratives.
  • Lack of Transparency is a Major Issue: The paper criticizes the lack of detailed documentation from Google and Apple regarding their training data and methodologies, contrasting with Twitter's more open approach.

Key Results

  • Male-Gaze Prevalence: 138 out of 336 images in the curated dataset exhibited male-gaze-like artifacts, resulting in an overall prevalence rate of approximately 41% (95% CI: 0.36-0.46). The prevalence varied significantly across albums, ranging from 19% to 79%.
  • Racial/Gender Bias: In the synthetic dataset, the erasure rates of faces of self-identified Black individuals were significantly higher under certain conditions. For example, in the WM-BF combination, Black females were erased more often than White males.
  • Framework Comparison:
    • Twitter's framework showed a unique tendency towards male-gaze cropping.
  • Google's and Apple's frameworks also exhibited strong face-centric saliency but showed extreme negative bias for White-Female faces in the WMWF combination, a departure from Twitter's preference for WF faces.
  • Google's CROP_HINTS API sometimes resulted in "middle" crops where neither face in a 3x1 grid was fully captured.

Limitations and Future Work

  • Dataset Specificity: The male-gaze findings are based on a specific type of image (long, thin, full-body women, logo-littered backgrounds). The authors acknowledge that these results are preliminary and call for more rigorous testing across wider datasets and aspect ratios.
  • Algorithmic Brittleness: The authors emphasize that the specific survival ratios are highly susceptible to manipulation and "Goodhart's Law." Their primary contribution is the framework for assessing risks, not definitive metrics.
  • Future Work: The authors hope their work will inspire further audits of other saliency cropping frameworks (e.g., from Facebook and Microsoft) and encourage greater transparency and detailed documentation from companies like Google and Apple regarding their training data and methodologies. They also suggest extending these investigations to different aspect ratios and larger, more diverse datasets.

Summary updated

BibTeX
@inproceedings{birhane2022auditing,
  author = {Abeba Birhane and Vinay Uday Prabhu and John Whaley},
  title = {{Auditing Saliency Cropping Algorithms}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
}
Figure