Sexism in the Judiciary: The Importance of Bias Definition in NLP and In Our Courts

Noa Baker GillisACL 2021

TL;DR

This paper tackles the challenge of accurately detecting gender bias in legal texts using NLP. The authors propose novel methods for automatically generating word lists to represent biases, demonstrating they are more robust and consistent than previous approaches. Their findings reveal significant gender bias in U.S. case law, which has decreased in correlation with increased female workforce participation over the last century.

What Problem Does This Paper Address?

The legal system, particularly through its case law, plays a crucial role in shaping societal norms and precedents. Detecting and understanding gender bias within this domain is vital to ensure fairness and prevent the perpetuation of harmful stereotypes. While Natural Language Processing (NLP) offers tools for bias detection, existing methods, particularly those relying on manual creation of word lists (like the Word Embedding Association Test or WEAT), suffer from inconsistency and susceptibility to researcher bias. These methods often use arbitrary word groupings (e.g., "salary," "job," "boss" for employment bias) that lack standardization, leading to unreliable results. Furthermore, the "significance" of detected bias magnitudes is often not clearly defined, making it difficult to interpret findings. This paper aims to address these limitations by developing more robust and automated methods for bias detection in legal texts.

Methods

The authors propose two novel methods for automatically generating word lists to represent gender bias, which are then used as input for the WEAT framework.

  1. WLOR (Weighted Log-Odds Ratio) with Thematic Grouping:
    • First-Order Processing: They first use a "first-order sorting" algorithm, specifically the WLOR algorithm, to identify words that appear more frequently when discussing one gender versus another in their case law dataset. This involves creating lexical histograms from sentences sorted by gender (based on pronouns and names) and then applying WLOR.
  2. Thematic Grouping: The output of WLOR is a list of gendered words. The top 500 most gendered words (250 skewed towards men, 250 towards women) are then manually grouped into thematic categories (e.g., employment, family). These thematic word lists are used as inputs for WEAT.
  3. WLOR with K-Means Grouping:
    • First-Order Processing: This method also starts with the WLOR algorithm to identify gendered words.
  4. K-Means Clustering: Instead of manual thematic grouping, this approach uses the K-Means clustering algorithm to automatically group the gendered words based on their vector representations (obtained from a Word2Vec model trained on the dataset). This aims to remove researcher bias from the grouping process. The resulting clusters are then filtered to include only the top 500 most gendered words and used as inputs for WEAT.

Both methods are tested for "base pair stability" (consistency across different gendered word pairs and capitalization) and compared against existing bias word lists.

Datasets

  • Case Law Access Project (CAP): The primary dataset used is the CAP, containing over 6.7 million U.S. state case decisions released by Harvard Law. This dataset spans a significant historical period, allowing for temporal analysis.
  • U.S. Census Data: Used in conjunction with the case law data for temporal analysis, specifically to track women's participation in the workforce over the last 100 years.

Key Insights and Ideas

  • Inconsistency of Manual Grouping: Existing methods that rely on manually curated word lists for bias detection are inherently inconsistent and susceptible to researcher bias, leading to unreliable results.
  • Automated Grouping for Robustness: Automatically generating word lists, either thematically or through clustering, significantly improves the robustness and consistency of bias detection.
  • Gendered Language in Law: U.S. case law exhibits significant gender bias, with distinct linguistic patterns associated with male and female subjects.
  • Correlation Between Bias and Workforce Participation: The decrease in gender bias in case law over time closely mirrors the increase in women's participation in the workforce.
  • Legal Language and Gender: Men are overwhelmingly associated with legal language and terms related to work and business, while women are more frequently associated with family-related terms. This suggests a potential implicit bias in how the law views gender roles.

Key Results

  • Improved Robustness: Both proposed methods (WLOR with thematic grouping and WLOR with K-Means clustering) passed the "base pair stability" test, indicating greater consistency than popular bias word lists from previous research.
  • Significant Gender Bias Detected: The study found statistically significant gender bias in case law. For example, employment terms were found to be biased against women with a magnitude of -0.19, and family terms were biased against men with a magnitude of 0.22. These findings were deemed significant based on the calculated mean and standard deviation of gender slant in the embedding space.
  • Temporal Trend: Over the last 100 years, the magnitude of gender bias in case law has decreased. This decrease is highly correlated with the increase in women's participation in the workforce, with R² values of 0.8781 for thematic grouping and 0.8676 for K-Means clustering.
  • Specific Word Associations: The word "office" was the most male-skewed word in U.S. case law, while "husband" was the most female-skewed. Legal language and terms related to work were heavily skewed towards men, with very few employment terms appearing in female-associated language until much later in the 20th century.

Limitations and Future Work

  • Limitations:
    • The study acknowledges that while their methods are more robust, the definition of "significance" for bias magnitude is based on a statistical threshold (one standard deviation above the mean) and could be further refined.
  • The authors note that their research focuses specifically on gender bias and does not explore intersections with other identities (e.g., race, ethnicity).
  • The study raises questions about whether the law implicitly views women as less independent legal entities, particularly in tort law, where female defendants are common but may face discrimination.
  • Future Work:
    • Intersectionality: Expand the research to explore bias against other demographic groups and the intersection of multiple identities (e.g., women of color).
  • Holistic Bias Clustering: Involve gender experts in developing more holistic approaches to bias clustering.
  • Impact on NLP Systems: Analyze the impact of these identified biases on the overall performance of NLP systems and the potential harms they might cause in other fields.
  • Legal Recognition of Women: Further investigate the legal system's implicit recognition of women as independent legal entities, especially in areas like tort law.

Summary updated

BibTeX
@inproceedings{gillis2021sexism,
  author = {Noa Baker Gillis},
  title = {{Sexism in the Judiciary: The Importance of Bias Definition in NLP and In Our Courts}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure