TL;DR
This paper investigates how wikiHow's how-to guides adapt content and style for different target audiences (e.g., gender, age). The authors build a new dataset, wikiHowAudiences, and demonstrate that computational models can effectively predict a guide's target audience based on its linguistic features, revealing subtle biases in how information is presented.
What Problem Does This Paper Address?
Instructional texts, like how-to guides, are designed to be accessible and effective for their intended readers. However, tailoring content to specific audiences, while aiming for clarity, can inadvertently introduce or reinforce societal biases and stereotypes. The authors highlight that platforms like wikiHow, with its vast collection of collaboratively edited guides, present a unique opportunity to study these audience-specific adaptations. While previous NLP research has used wikiHow for tasks like summarization and goal-step inference, none have explicitly investigated how guides differ based on target audience characteristics, nor have they systematically analyzed the potential for bias introduced by these adaptations. This paper aims to fill this gap by exploring the extent to which wikiHow guides exhibit audience-specific differences and whether these can be computationally identified.
Methods
The researchers employed a multi-pronged approach:
- Corpus Construction: They created a new dataset, wikiHowAudiences, by filtering the existing wikiHowToImprove dataset for guides with audience-specific indicators (e.g., "(Girls)", "(Boys)"). This corpus focuses on four primary audience groups: Women (W), Men (M), Kids (K), and Teens (T).
- Case Studies:
- Same Title, Different Audience: They analyzed pairs of guides with identical titles but different audience indicators to identify differences in topic, length, style, and content. This involved manual topic categorization and quantitative analysis of article length and word overlap (using BLEU score).
- "How To Be" Guides: They specifically examined guides starting with "how to be" to understand how goals and desired outcomes are presented differently across audiences.
- Generalization Study (Computational Approach): To assess if the observed differences were systematic, they developed computational models. This involved:
- Feature Engineering: Creating features based on content (uni-grams, bi-grams from titles and articles), style (lexical and syntactical features), and article length.
- Language Model Embeddings: Utilizing pre-trained language models (RoBERTa) to generate article representations.
- Classification: Training logistic regression models and using RoBERTa embeddings to classify articles into target audience groups (W-M and K-T).
Datasets
- wikiHowToImprove: The base dataset used for constructing the new corpus.
- wikiHowAudiences: A newly constructed sub-corpus derived from wikiHowToImprove, specifically curated to include how-to guides with audience-specific indicators. This dataset is made publicly available.
Key Insights and Ideas
- Systematic Differences Exist: The studies reveal that wikiHow guides are not monolithic; they exhibit systematic differences in topic, style, and content when tailored for different audiences.
- Length Variation: Instructions for Kids (K) are significantly shorter than those for Teens (T), suggesting a difference in expected reading comprehension or attention span.
- Subtle Biases in Content and Style: Even when guides share the same title, subtle differences in vocabulary, punctuation, and emphasis can be observed. For example, "Get Clear Skin" for women might include more specific information than a similar guide for men.
- "How To Be" Guides Highlight Goal Differences: Guides instructing readers "how to be" reveal distinct goals and desired outcomes for different genders. For instance, women's guides might focus on being "cute" and "popular," while men's guides emphasize being "cool" and "more."
- Computational Models Can Detect Audience: The generalization study demonstrates that machine learning models can accurately predict a guide's target audience using linguistic features, indicating that these audience-specific characteristics are linguistically encoded.
- Stereotypes as Predictive Features: The analysis of predictive features for classification models revealed the use of stereotypical language (e.g., "cute," "makeup" for women; "man," "boy" for men) and gender-inclusive language ("theirs" for M), as well as direct address ("kid" for K, "teen" for T). Worryingly, negations were also found to be strong features for women, which can reinforce stereotypes.
Key Results
- Classification Performance: The combined model (content, style, and length features) achieved the best macro F₁-scores, reaching 0.71 for W-M and 0.78 for K-T on the test set. RoBERTa-based models also performed competitively.
- Feature Importance: Content and style features, particularly at the article level, were found to be highly predictive of the target audience.
- "Same Title" Accuracy: The combined model correctly classified 75% of the "same title, different audience" articles, suggesting that audience-specific adaptations are often discernible even within closely related content.
- Gender Imbalance: The corpus construction revealed that women are mentioned as a target audience more than four times more frequently than men, and teens receive about 50% more instructions per article than kids.
Limitations and Future Work
- Language Specificity: The study is limited to English-language guides, and findings may not generalize to other languages.
- Data Source Limitation: Relying solely on wikiHow means the identified audiences and their portrayals might not be exhaustive or representative of all societal diversity, particularly marginalized groups.
- Intersectionality: The current analysis primarily considers single attributes (gender, age). Future work should explore intersectionality, as real-world readers possess multiple coexisting attributes.
- Cultural Bias: The content primarily reflects Western cultural perspectives.
- Future Work: The authors suggest expanding the dataset with more diverse sources and languages, exploring intersectional analyses, and developing methods to mitigate identified biases. They also aim to foster interdisciplinary dialogue to raise awareness and address these inequalities.
Summary updated