On Shortcuts and Biases: How Finetuned Language Models Distinguish Audience-Specific Instructions in Italian and English

Nicola Fanton, Michael RothACL 2024

TL;DR

This paper investigates how fine-tuned language models learn to distinguish audience-specific instructions in Italian and English wikiHow articles. By analyzing model attributions, they reveal that models rely on group terms, grammatical gender, and stereotypical attributes, highlighting potential biases that can be learned and amplified by these models, even in a less-resourced language like Italian.

What Problem Does This Paper Address?

Instructional texts, like those found on wikiHow, are tailored to specific audiences to improve clarity and relevance. However, this tailoring can inadvertently perpetuate societal biases, such as gender stereotypes. While previous research has explored these biases in English wikiHow articles using simpler methods, there's a gap in understanding how these biases manifest in other languages, particularly less-resourced ones like Italian, and whether more sophisticated models like fine-tuned language models (LMs) exacerbate these issues. This paper aims to bridge this gap by analyzing audience-specific instructions in both English and Italian, using advanced LM interpretation techniques.

Methods

The authors employ a two-pronged approach:

  1. Data Analysis: They first perform a topic-based analysis of their newly created Italian wikiHow dataset (wikiHowAudIT) and a previously existing English dataset (wikiHowAudiences). This involves using agglomerative clustering on sentence embeddings to group articles by topic and then analyzing the highest-scoring tokens within each cluster to identify overarching themes.
  2. Model-Based Analysis: For a deeper dive into how models learn to distinguish audiences, they fine-tune various language models (both monolingual and multilingual) for binary classification tasks (e.g., Women vs. Men, Kids vs. Teens). To understand what these models learn, they utilize the Integrated Gradients method. This technique attributes importance scores to individual tokens in the input text, revealing which words or phrases are most influential in the model's decision to classify an article for a specific audience.

Datasets

  • wikiHowAudIT (WHA-IT): A newly compiled dataset of Italian wikiHow articles, specifically focusing on those with audience indicators in their titles.
  • wikiHowAudiences (WHA-EN): An existing English dataset of wikiHow articles, previously used in related work.

Key Insights and Ideas

  • Cross-lingual Similarities in Bias: Despite language differences, models consistently learn to distinguish audiences using similar shortcuts: group terms (e.g., "men," "girls"), grammatical gender markers, and stereotypical attributes.
  • Italian's Grammatical Gender as a Shortcut: Italian's rich grammatical gender system provides a particularly strong signal for models to differentiate between male and female-targeted content, potentially making it easier for models to learn these distinctions.
  • Model Performance Varies by Task and Language: While models perform well on the Italian W-M task, performance drops significantly for the Italian K-T task, possibly due to less distinct audience indicators or smaller data size. English tasks generally show higher performance.
  • Multilingual Models Underperform: Monolingual models (Italian for Italian data, English for English data) consistently outperform multilingual models, suggesting that cross-lingual training might not be beneficial for this specific task and language pair.
  • Stereotypes Persist: The analysis confirms that models pick up on and leverage stereotypical associations, such as "makeup" for women and "handsome" for men, which can reinforce existing societal biases.

Key Results

  • Classification Performance: Fine-tuned LMs achieve high macro F1 scores on the test sets, with W-M tasks generally outperforming K-T tasks across both languages. For WHA-IT, the best W-M score is 0.87, and for K-T, it's 0.60. For WHA-EN, the scores are 0.84 (W-M) and 0.78 (K-T).
  • Attribution Analysis:
    • Group Terms: Models heavily rely on explicit group identifiers (e.g., "men," "guy," "boy" in English; "uomini," "uomo," "ragazzo" in Italian). Italian shows fewer such terms in top attributions compared to English.
  • Grammatical Gender: In Italian, grammatical gender (e.g., "sicura" for feminine "sure" vs. "sicuro" for masculine "sure") serves as a strong distinguishing feature.
  • Stereotypes: Attributions reveal stereotypical associations, like "makeup," "pretty," "pink" for women and "gentleman," "nerd," "handsome" for men in English.

Limitations and Future Work

  • Cultural Scope: The study is limited to "western culture" and does not account for intersectional perspectives beyond gender and age.
  • Data Source: The reliance on a single data source (wikiHow) limits generalizability.
  • Future Work: The authors suggest exploring other languages and cultures, masking identified shortcuts to see if models can learn more nuanced distinctions, and focusing on identifying and removing inadequate biases while retaining useful audience adaptations. They also propose investigating how to mitigate these learned biases.

Summary updated

BibTeX
@inproceedings{fanton2024shortcuts,
  author = {Nicola Fanton and Michael Roth},
  title = {{On Shortcuts and Biases: How Finetuned Language Models Distinguish Audience-Specific Instructions in Italian and English}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure