Introduction
Language often harbors implicit social biases through subtle framing, even in the absence of overt slurs. In narrative contexts, characters are frequently portrayed with varying levels of "agency"—the capacity to act intentionally. Research indicates that female characters are often described with low-agency verbs (e.g., "waited," "daydreamed"), whereas male characters are associated with high-agency actions (e.g., "pursued," "decided").
Addressing this "portrayal bias" is challenging due to the lack of parallel "biased-to-unbiased" datasets and the need to preserve narrative meaning while altering pragmatic framing. PowerTransformer is introduced as an unsupervised framework to rewrite text, transforming character agency without requiring direct supervision.
Methodology
PowerTransformer utilizes an encoder-decoder architecture based on OpenAI's GPT. The framework operates without parallel data by integrating three key components:
1. Connotation Frames
The model leverages a predefined lexicon that assigns agency scores to verbs. This allows the system to identify the pragmatic power dynamics inherent in specific actions.
2. Joint Training Objective
The model is trained simultaneously on two tasks:
- Reconstruction (Denoising): The model learns to fill in masked verbs given a "target agency" token (e.g.,
<Pos>for high agency), effectively learning the relationship between context and agency. - Paraphrasing: By training on an external paraphrase corpus, the model learns to maintain semantic consistency during the revision process.
3. Vocab Boosting at Inference
A novel "Vocab Boosting" technique is applied during decoding. Instead of relying solely on learned weights, the model increases the logit scores of verbs that align with the desired agency level. This steers the generation toward appropriate "high-power" or "low-power" words that fit the grammatical and narrative context.
Datasets and Evaluation
The model was trained and tested using several diverse datasets:
- ROC Story Corpus: 100,000 five-sentence stories for the reconstruction task.
- OpusParcus: A large paraphrase dataset derived from movie and TV subtitles.
- Movie Scripts Corpus: 767 modern English movie scripts used to evaluate the model's ability to mitigate real-world gender bias.
Key Results
PowerTransformer demonstrates significant improvements over baseline models:
- Target Accuracy: The model achieved 89% accuracy in reaching the target agency level, far surpassing PPLM (13%).
- Meaning Preservation: It maintained a high 0.96 BERTScore, ensuring that revisions remained faithful to the original story logic.
- Human Preference: In head-to-head evaluations, human judges preferred PowerTransformer’s revisions for both agency-matching and semantic preservation.
- Bias Mitigation: When applied to movie scripts, the model successfully adjusted female character portrayals to agency levels comparable to male characters.
Limitations and Future Work
While effective, the model's reliance on a fixed lexicon limits its performance on out-of-vocabulary verbs. Additionally, the current implementation uses coarse agency categories (Positive, Neutral, Negative) and focuses on binary gender portrayals. Future research may explore more granular social dimensions and broader social constructs.
Summary updated