When natural language processing (NLP) systems process human attributes, they typically operate under a strict, systemic assumption: that gender is a mutually exclusive, binary variable. This binary gender constraint is embedded at every level of modern language technologies, from the tokenizers and embedding spaces to training datasets and evaluation benchmarks.
While treating gender as a simple binary choice simplifies model evaluation, it introduces a profound representational gap. When language models encounter non-binary identities, neopronouns, or gender-neutral expressions, their internal mechanics break down. Understanding why this constraint exists—and how it distorts model behavior—requires looking under the hood at how language models resolve linguistic ambiguity.
The Roots of the Binary Constraint
The binary constraint in language technologies is driven by two primary forces: the morphosyntactic structures of target languages and the design of early evaluation benchmarks.
In morphologically rich languages like Spanish, French, Italian, and Arabic, gender is not just a property of pronouns; it is a structural requirement that propagates through entire sentences via agreement chains. Adjectives, nouns, and verbs must be inflected to match the gender of the subject. As Beatrice Savoldi et al. demonstrate in an analysis of speech translation systems, these complex morphosyntactic agreement chains force models to make explicit grammatical choices (Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation).
To measure how models handle these choices, pioneering benchmarks framed gender fairness as a balanced binary equation. For example, the WinoMT dataset introduced by Gabriel Stanovsky et al. evaluates whether a machine translation system can correctly translate ambiguous occupations (like "the doctor" or "the nurse") into both masculine and feminine target forms based on contextual pronouns (Evaluating Gender Bias in Machine Translation).
[Source Text] "The doctor text message her patient."
│ │
└───────┬─────────┘
▼
[Target Text] "La médica envió un mensaje..." (Feminine Agreement Chain)
▲ ▲
│ │
└─────┴── Model must correctly inflect both article and noun
While these benchmarks were vital for exposing how models default to harmful societal stereotypes, they established an evaluation paradigm where fairness is achieved simply by balancing masculine and feminine outputs. This paradigm effectively hardcoded a binary "folk model" of gender into NLP research, omitting non-binary individuals and gender-neutral linguistic strategies entirely.
Where the Binary Model Breaks Down
When a system trained under the binary constraint encounters non-binary language, it experiences systemic failures rather than simple translation errors.
Entity Erasure in Coreference Resolution
Coreference resolution systems are designed to determine which pronouns refer to which nouns in a text. When these systems encounter non-binary pronouns (such as the singular they or neopronouns like xe/xem), they frequently fail to recognize them as referential words at all.
Yang Trista Cao et al. exposed this mechanical failure by introducing the GICoref dataset (Toward Gender-Inclusive Coreference Resolution). The authors showed that because state-of-the-art models assume gender is an immutable binary attribute, their transition matrices and attention mechanisms fail to link non-binary pronouns back to their corresponding entities. Instead of misgendering the subject, the model completely drops the connection, treating the pronoun as an unrelated token and erasing the individual from the text's semantic structure.
The Failure Mode of Neopronouns
In machine translation, the binary constraint creates a severe representational gap when processing identity-inclusive language. Dirk Hovy evaluated how commercial translation engines handle traditional pronouns versus neopronouns (What about "em"? How Commercial Machine Translation Fails to Handle (Neo-)Pronouns).
Because these models are trained on massive web-scraped datasets dominated by binary classifications, their tokenizers and embedding spaces lack stable representations for neopronouns. When forced to translate a sentence containing a neopronoun into a language with rigid grammatical gender, the system cannot map the token to a neutral category. It either defaults to a masculine inflection due to statistical prevalence or produces corrupted, ungrammatical text.
Amplifying the Masculine Default
When a source sentence is completely gender-neutral or ambiguous, models do not distribute their predictions evenly. Instead, they fall back on a "masculine default." As Beatrice Savoldi et al. observe in the MuST-SHE test suite, models consistently favor masculine inflections in the absence of explicit cues (Test Suites Task: Evaluation of Gender Fairness in MT with MuST-SHE INES).
This is not merely a passive reflection of real-world data distributions. Orfeas Menis Mastromichalakis et al. demonstrated that machine translation systems actively amplify stereotypes, defaulting to masculine forms for professional occupations even when real-world demographic data shows a clear female or gender-neutral majority in that field (Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Terms).
The Mechanics of Mitigation
To move past the binary constraint, researchers have developed techniques that alter how models encode and process gender information without requiring them to be retrained from scratch.
Domain Adaptation vs. Data Scrubbing
A naive approach to fixing gender bias is to "debias" or scrub the training data before a model is trained. However, Marcus Tomalin et al. argue that this approach is both practically and ethically flawed: it degrades overall translation quality and fails to address deep-seated architectural biases (The practical ethics of bias reduction in machine translation: why domain adaptation is better than data debiasing).
Instead, a more effective mechanism treats debiasing as a domain adaptation problem. As pioneered by Bill Byrne et al., this technique takes a fully trained, high-performing model and fine-tunes it on a tiny, meticulously balanced dataset of counterfactual examples (Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem). This allows the model to retain its vast linguistic knowledge base while rapidly shifting its inference behavior to respect diverse gender markers and neutral forms.
Latent Space Alignment via Contrastive Learning
To prevent models from relying on surface-level lexical shortcuts, researchers have targeted the geometry of the model's internal representations. Minwoo Lee et al. introduced Gender-Aware Contrastive Learning (GACL) to decouple semantic meaning from rigid gender assignments (Target-Agnostic Gender-Aware Contrastive Learning for Mitigating Bias in Multilingual Machine Translation).
Standard Latent Space:
[doctor] ─── (Closer to Masculine Tokens) ─── [he/him]
[nurse] ─── (Closer to Feminine Tokens) ─── [she/her]
GACL Latent Space (Clustered by Contextual Gender):
[Context: Non-binary] ───► Clustered Together ───► [doctor] [nurse] [they/them]
[Context: Feminine] ───► Clustered Together ───► [doctor] [nurse] [she/her]
GACL works by forcing the model's encoder to cluster internal representations based on the contextual gender of a sentence rather than the specific nouns used. By optimizing a contrastive loss function, the model learns to group sentences with similar gender contexts together in the latent space, regardless of the target language. This geometric alignment allows the model to generalize debiasing strategies to multiple languages simultaneously, breaking the rigid association between specific occupations and binary gender inflections.
Inference-Time Context Steering
Another highly effective mechanism operates entirely during inference, bypassing fine-tuning altogether. Shanya Sharma et al. demonstrated that neural machine translation models can be dynamically "steered" by prepending explicit contextual instructions or hand-crafted templates to the input text (How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contexts).
By injecting a clear, non-ambiguous context sentence (e.g., specifying the non-binary or specific gender identity of the subject in the preceding phrase), the model's attention heads are guided to select the correct morphological inflections in the target language. This approach shifts the burden of gender resolution away from historical training statistics and onto the immediate, explicit context provided at runtime.
The Next Frontier: Automated Gatekeepers
While mitigation strategies are advancing, the tools used to evaluate them remain a bottleneck. Automated Quality Estimation (QE) metrics serve as the gatekeepers for deploying language models in production, deciding which translations are correct.
However, these evaluation metrics are themselves trapped within the binary paradigm. Recent audits of state-of-the-art QE metrics reveal that they systematically penalize gender-neutral language and inclusive phrasing, while favoring traditional masculine defaults. Even when a model successfully breaks the binary gender constraint to produce an inclusive translation, the automated gatekeeper may flag it as an error.
Overcoming the binary constraint requires a comprehensive overhaul of the entire NLP pipeline. Until evaluation metrics, tokenizers, and datasets are intentionally designed to accommodate non-binary, epicene, and gender-neutral expressions, language technologies will continue to enforce an artificial binary on a diverse linguistic world.
Key Foundational Work
Evaluating Gender Bias in Machine Translation (2019) — contributed 1 concept elements to this direction.
Developing Age and Gender Predictive Lexica over Social Media (2014) — contributed 1 concept elements to this direction.