Abstract
While hyperbolic word embeddings like Poincaré GloVe excel at capturing hierarchical data, they inherit and potentially amplify gender biases from training corpora. This paper introduces gyrocosine bias, a novel metric for non-Euclidean spaces, and proposes Poincaré Gender Debias (PGD). PGD is a post-processing method that mitigates stereotypes while maintaining the mathematical advantages of hyperbolic geometry. Evaluation shows that PGD can reduce stereotype scores to zero while preserving the semantic utility of the original embeddings.
What Problem Does This Paper Address?
Word embeddings serve as the mathematical foundation for modern Natural Language Processing (NLP). However, they are notorious for "learning" human prejudices from training data—for instance, associating "doctor" with men and "nurse" with women. While extensive research has addressed debiasing in Euclidean spaces (e.g., standard GloVe), the field has recently shifted toward hyperbolic space for its superior ability to represent hierarchical relationships (like "canine" $\rightarrow$ "dog" $\rightarrow$ "poodle").
Until this research, there was no established method to measure or mitigate gender bias in these curved, non-Euclidean geometries. The authors demonstrate that hyperbolic models not only inherit these biases but may actually amplify them due to the negative curvature of the space.
Methods
The authors bridge the gap between Euclidean debiasing and hyperbolic geometry using Gyrovector Space Theory.
- Gyrocosine Bias: In Euclidean space, bias is measured via cosine similarity. In hyperbolic space, standard angles are insufficient. The authors propose "gyrocosine bias," utilizing Möbius addition and gyrovectors to calculate the differential association of neutral words (e.g., "manager") with gendered definitional words.
- Poincaré Gender Debias (PGD): This post-processing optimization routine solves a multi-objective problem for every word in the vocabulary:
- Fairness: Minimize the difference in a word's association with male and female gender vectors.
- Utility: Maximize the similarity between the original biased vector and the new debiased vector to preserve semantic meaning.
- Optimization: Because the Poincaré ball is a Riemannian manifold, the authors employ Riemannian ADAM, a specialized optimizer that respects the space's curvature during debiasing.
Key Results
The proposed PGD method achieves significant bias reduction across multiple benchmarks without sacrificing model performance.
- WEAT Improvements: In the "Math vs. Arts" category, the $p$-value (where higher indicates less significant bias) increased from 0.30 in the biased model to 0.82 after PGD.
- Analogy Accuracy: On the SemBias test, PGD improved "Definitional" analogy accuracy from 75.3% to 83.0%. Most notably, the "Stereotype" score dropped to 0.0, meaning the model completely stopped preferring biased analogies like "man is to doctor as woman is to nurse."
- Minimal Semantic Loss: Performance across semantic similarity benchmarks (e.g., SimLex-999) remained nearly identical to the original biased model, proving that bias can be removed without compromising the underlying word representations.
Conclusion and Future Work
This work establishes the first framework for fairness in hyperbolic embeddings. While PGD is a successful post-processing step, the authors suggest that future research should explore "debiasing-by-design," integrating fairness constraints directly into the initial training loss. Additionally, while this study focused on binary gender, extending these hyperbolic metrics to non-binary identities and other social biases (such as race or religion) remains a critical next step for the field.
Summary updated