Abstract
Gender bias is one of the most common and well-studied demographic biases in information retrieval, and in general in AI systems. After discovering and reporting that gender bias for certain professions could change searchers' worldviews, mainstreaming image search engines, such as Google, quickly took action to correct and fix such a bias. However, given the nature of these systems, viz., being opaque, it is unclear if they addressed unequal gender representation and gender stereotypes in image search results systematically and in a sustainable way. In this paper, we propose adversarial attack queries composed of professions and countries (e.g., 'CEO United States') to investigate whether gender bias is thoroughly mitigated by image search engines. Our experiments on Google, Baidu, Naver, and Yandex Image Search show that the proposed attack can trigger high levels of gender bias in image search results very effectively. To defend against such attacks and mitigate gender bias, we design and implement three novel re-ranking algorithms -- epsilon-greedy algorithm, relevance-aware swapping algorithm, and fairness-greedy algorithm, to re-rank returned images for given image queries. Experiments on both simulated (three typical gender distributions) and real-world datasets demonstrate the proposed algorithms can mitigate gender bias effectively.
Introduction: The Illusion of Fairness
Image search engines are more than just tools; they are mirrors that reflect and shape our perception of reality. Past research famously highlighted that a search for "CEO" yielded results dominated by white men, leading tech giants like Google and Bing to implement fixes. However, these fixes often appear to be "hit-or-miss" rather than systematic. Because search algorithms are opaque, it is unclear if they have truly addressed underlying stereotypes or simply hard-coded a more diverse set of results for a few high-profile keywords.
The "Band-Aid" Fix: Adversarial Attacks
To test the robustness of these fixes, the authors designed adversarial queries that combine an occupation with a country (e.g., "CEO United States"). This simple modification bypasses the manual "fixes" applied to the keyword "CEO" and triggers the underlying biased ranking logic.
As shown in Figure 1, the query "CEO" returns a diverse set of images. However, as seen in Figure 2, appending "United States" causes the results to revert to a predominantly male distribution. This suggests that search engines are using "superficial mitigation" for specific high-traffic queries rather than fixing the core ranking bias.
Methodology
The researchers developed the Cross-search-engine Image Retrieval Framework (CIRF) to collect over 18,000 images from four global engines: Google (USA), Baidu (China), Naver (South Korea), and Yandex (Russia). To detect gender in "in the wild" search images, they employed a hybrid approach combining automated detection with crowdsourced human verification via Amazon Mechanical Turk.
Mitigation Algorithms
The paper proposes three re-ranking strategies to systematically improve fairness:
- Epsilon-greedy: Randomly swaps images in the list with a controllable degree of randomness ($\epsilon$) to break up biased clusters.
- Relevance-aware Swapping: Uses a "relevance weight" based on the original rank to ensure highly relevant images are less likely to be moved to the bottom.
- Fairness-greedy: Identifies the most underrepresented gender based on a "ground truth" (like census data) and moves the first available image of that gender up the list to minimize the distribution gap.
Key Results and Insights
- Attack Success: Adversarial queries successfully triggered high levels of gender bias across all four search engines. Bias isn't unique to Western engines; it was observed across Baidu, Naver, and Yandex, though patterns varied by culture.
- Algorithm Performance: In experiments on Google's "biologist U.S." results, the Fairness-greedy algorithm reduced the bias score (KL divergence) from 0.138 to 0.018, a massive improvement.
- Relevance vs. Fairness: There is an inherent trade-off, but the authors show that significant fairness improvements are possible with only minor adjustments to the original ranking, preserving the utility of the search engine for the user.
Conclusion
While major search engines have "patched" gender bias for high-profile queries, these fixes are often superficial. This study highlights the need for systematic, algorithmic solutions. Future work should address the limitations of binary gender classification and explore intersectional biases involving race and age, which likely compound the observed disparities.
Summary updated