TL;DR
Mem2Fair is a plug-and-play framework that mitigates demographic bias in text-to-video (T2V) models without requiring internal model access or expensive retraining. By maintaining a "Fair Memory" of past outputs and using an LLM to subtly optimize future prompts, it steers video generation toward target demographic distributions in a closed-loop, online fashion.
What Problem Does This Paper Address?
Text-to-video models, like their text-to-image (T2I) predecessors, often reflect societal biases found in their training data. If you prompt a model for a "nurse" without specifying gender, it might disproportionately generate videos of women.
While many fairness interventions exist for images, they fail in the video domain for three main reasons:
- Temporal Fidelity: Intervening in the model's latent space or denoising process often breaks temporal consistency, leading to flickering or "identity shifting" across frames.
- Model Access: Many state-of-the-art T2V models (like Kling or Sora) are closed-source. Most existing fairness methods require access to the model's internal weights or attention maps.
- Sampling Cost: Generating video is computationally expensive. Methods that require thousands of "calibration" samples before deployment are often prohibitively costly.
The authors tackle these gaps by treating fairness as a conditional online approachability problem—essentially a control system that adjusts its behavior based on real-time feedback.
Methods
Mem2Fair operates as a wrapper around any T2V generator. It consists of four main components:
- Concept-Conditioned Fair Memory: The system maintains a separate "memory" for different semantic concepts (e.g., "doctor," "firefighter"). This memory tracks the running tally of demographic outcomes (gender, race, age) for that specific concept.
- Online Approachability Controller: When a new prompt arrives, the controller looks at the current state of the Fair Memory. It calculates the "fairness gap" (the discrepancy between the current empirical distribution and the desired target) and selects an attribute value (e.g., "male") that would best help close that gap.
- Prompt Optimization: An LLM (specifically Qwen3-0.6B in their experiments) takes the original prompt and the selected attribute, then rewrites it to be explicit while preserving the original context. For example, "A doctor walking in a hospital" becomes "A female doctor walking in a hospital."
- Outcome Feedback Loop: After the video is generated, an external attribute classifier (like DeepFace) analyzes the result. This feedback is fed back into the Fair Memory to inform the next generation.
Crucially, this is an online method: it learns and adjusts as it goes, requiring no pre-deployment training or offline sampling.
Datasets
The researchers evaluated Mem2Fair using:
- Stable Bias Dataset: Used to derive a set of 50 in-distribution and 25 held-out "unseen" occupation concepts.
- O*NET 31.0: Used to construct complex, context-rich prompts for robustness testing.
- Base Models: Evaluation was performed on Wan2.2-5B (open-source) and commercial black-box APIs including Seedance 2.0 and Kling v2.6.
Key Insights and Ideas
- Fairness as Control Theory: The most elegant shift here is moving away from "fixing the model weights" to "steering the model inputs." By formulating fairness through Blackwell approachability, the authors provide a mathematical guarantee that the system will converge toward the target distribution over time.
- Concept-Level Granularity: Bias is not uniform. A model might be biased toward men for "engineers" but biased toward women for "nurses." By using concept-conditioned memory, Mem2Fair applies surgical interventions only where they are needed.
- Preserving the "Black Box": Because Mem2Fair only interacts with the prompt (input) and the video (output), it is entirely agnostic to the underlying architecture. This makes it one of the few viable fairness solutions for the current era of proprietary API-based models.
- Temporal Safety: By intervening at the prompt level rather than the latent level, the "motion" and "consistency" logic of the T2V model remains untouched, ensuring high-quality video.
Key Results
The authors compared Mem2Fair against several T2I baselines adapted for video (FairGen, FairImagen, LightFair).
- Fairness Gains: On the Wan2.2 model, Mem2Fair improved Normalized Parity (NP) from 0.336 to 0.917 (where 1.0 is perfect fairness). The Bias Score dropped significantly from 0.384 to 0.052.
- Quality Preservation: Unlike some baselines that degraded video quality to achieve fairness, Mem2Fair maintained comparable VBench scores (measuring motion smoothness and subject consistency) and CLIPsim (prompt-video alignment) to the vanilla model.
- Commercial Success: On Kling v2.6, a leading commercial model, Mem2Fair raised the NP from a very low 0.122 to 0.864, proving its effectiveness on real-world, high-imbalance systems.
- Generalization: The system successfully improved fairness even for "unseen" occupations not included in its initial vocabulary.
Limitations and Future Work
- Classifier Reliability: The system is only as good as its feedback. If the external face/attribute classifier is itself biased or fails to detect a face (e.g., due to low lighting or profile views), the Fair Memory will not update correctly.
- The "Cold Start" Problem: For a brand-new concept with zero history, the first few videos will still reflect the base model's inherent bias until the memory accumulates enough observations to trigger a correction.
- Discrete Attributes: The current framework focuses on discrete categories (e.g., Male/Female). Extending this to non-binary attributes or continuous demographic scales remains an open challenge.
- Prompt Optimization Latency: Adding an LLM call into the generation pipeline adds a small amount of overhead, though the authors used a lightweight 0.6B parameter model to minimize this.
Summary updated