TL;DR
While raw data suggests men receive harsher moral judgments online, a new causal inference study reveals this isn't due to direct gender bias. By matching highly similar stories from Reddit's /r/AITA community, researchers found that the discrepancy is almost entirely explained by the different types of situations men and women choose to share.
What Problem Does This Paper Address?
Correlational studies frequently show that male protagonists in moral dilemmas are judged more harshly than females. However, correlation isn't causation. Does this reflect a deep-seated gender bias among online judges, or do men simply land themselves in—and choose to post about—more clear-cut violations of social norms?
Disentangling this is crucial for understanding human decision-making and ensuring that large language models (LLMs), which are heavily trained on internet data like Reddit, don't inherit and amplify false biases. The authors set out to isolate the direct causal effect of a protagonist's declared gender on the moral judgments they receive.
Methods
The authors constructed a causal framework to map out direct vs. mediated effects of gender on moral judgment.
To isolate the direct effect (the dashed line in Figure 1), they used Propensity Score Matching (PSM) with calipers to pair stories describing nearly identical situations but featuring different protagonist genders. They leveraged SBERT (all-mpnet-base-v2) to generate text embeddings for semantic similarity and trained a BERT model as a propensity scorer to predict the author's gender based on the text.
Crucially, they implemented a "gender neutralization" strategy during training—swapping gendered words like "wife" and "husband" with a 50% probability—to prevent the model from relying on lexical shortcuts. Latent Dirichlet Allocation (LDA) was used to cluster posts into six distinct topics to control for context. Finally, they calculated the Sample Average Treatment Effect on the Treated (SATT) across different semantic distance thresholds.
Datasets
The study utilized a massive scrape of the popular Reddit community /r/AITA (Am I The Asshole) spanning 2014 to 2020.
- Raw Dataset: 250,770 submissions and over 6.8 million comments.
- Core Analytical Dataset: 33,421 submissions that contained explicit, self-disclosed demographic tags (e.g., "M22", "F26") and received a clear community judgment (blame vs. no blame) weighted by comment scores.
Key Insights and Ideas
- The "Situation" Mediates the Bias: The apparent harshness toward men is a reporting/behavioral artifact. The situation itself acts as a full mediator; men are simply more likely to share stories where they exhibit higher agency or engage in riskier behavior that violates social norms.
- Context Matters: When broken down by topic, a small but statistically significant bias against men does persist in one specific domain: "friendship and relationships." This suggests gender roles and stereotypes exert a stronger influence in specific social contexts.
- Loose Matches Confound Causal Claims: Human evaluation revealed that when semantic matching is too loose, the crude "male-blame" correlation leaks back in. Tight semantic constraints are mandatory for valid text-based causal inference.
Key Results
- In the unadjusted dataset, men were roughly twice as likely as women to receive a negative judgment (Odds Ratio = 2.21).
- After applying tight propensity score matching, the SATT dropped to near zero and lost statistical significance for closely matched pairs.
- As seen in Figure 2a, the overall SATT hovers around zero at low semantic distances but creeps upward as the matching threshold loosens (introducing lower-quality matches).
- When stratified by topic (Figure 2b), only "friendship and relationship" showed a distinct, statistically significant positive SATT, indicating a negative bias toward men unique to that context.
Limitations and Future Work
The study is restricted to text-mediated, pseudonymous online interactions on a platform with a specific demographic skew (young, female, and US-dominated). It also assumes a homogeneous jury due to Reddit's anonymity constraints, meaning unobserved confounders like judge homophily (e.g., male judges being more lenient toward male authors) could not be fully controlled.
The authors suggest that future work should focus on scaling the measurement of protagonist agency to further validate why men and women choose to share different types of morally ambiguous stories.
Summary updated