For the Misgendered Chinese in Gender Bias Research: Multi-Task Learning with Knowledge Distillation for Pinyin Name Gender Prediction

Xiaocong Du, Haipeng Zhang

Abstract

Global gender bias research often relies on automated tools that fail significantly when processing Chinese names in Pinyin, leading to the systematic exclusion or misgendering of Chinese individuals. This paper introduces a novel Multi-Task Learning (MTL) framework that utilizes Knowledge Distillation to "teach" a Pinyin-based model the rich semantic gender cues found in original Chinese characters (Hanzi). Our approach outperforms leading commercial tools by up to 20%, offering a more inclusive methodology for large-scale bibliometric and social science studies.

Introduction: The Exclusion Bias in Global Research

In large-scale social science and bibliometric studies, researchers use name-based gender inference tools (such as Namsor or GenderAPI) to assign gender labels when demographic data is missing. While these tools perform reliably for Western names, they are notoriously inaccurate for Chinese names written in Pinyin—the romanized phonetic system.

Error rates of commercial gender inference tools across different ethnicities.

As illustrated in Figure 1, the error rate for Chinese names is 7 to 12 times higher than for European names. Consequently, many researchers simply exclude Chinese participants from their datasets to maintain data quality. This creates a massive "exclusion bias," which is particularly problematic as Chinese authors now contribute nearly 20% of international scientific papers.

The Challenge: The Semantic Gap of Pinyin

The core of the problem is that Pinyin is a "many-to-one" mapping. A single Pinyin string like yan could correspond to dozens of different Chinese characters (Hanzi). In Hanzi, gender is often encoded semantically—for instance, the character for "beautiful" (妍) is traditionally female, while "flame" (炎) is traditionally male. Once converted to Pinyin, this semantic information is lost, leaving models with only ambiguous phonetic data.

Methodology: Bridging the Gap with Knowledge Distillation

To address this information loss, we propose a student-teacher framework designed to bridge the gap between Pinyin and Hanzi.

Architecture of the proposed Multi-Task Learning framework with Knowledge Distillation.

1. Syllable-Aware Preprocessing

Standard BERT tokenizers often break Pinyin incorrectly (e.g., splitting "jianguo" into "ji," "ang," and "uo"). We utilize a specialized "Pinyinsplit" tool to ensure the model processes syllables that correspond to actual Chinese characters ("jian" and "guo").

2. Multi-Task Learning (MTL)

The student model (the Pinyin predictor) is trained on two tasks simultaneously:

  • Primary Task: Predicting gender from Pinyin.
  • Auxiliary Task: Predicting the most likely underlying Chinese characters. This forces the model to learn the latent associations between sounds and meanings.

3. Knowledge Distillation (KD)

This is the central innovation. We train a "Teacher" model on original Hanzi, which possesses much higher accuracy due to its access to semantic radicals. We then use Feature-based and Response-based distillation to transfer this "semantic intuition" to the Pinyin-based "Student" model. Essentially, the student learns to recognize which Pinyin strings are likely to represent gendered characters.

Datasets and Experimental Setup

The study utilizes three distinct datasets to ensure robustness:

  • 9,800 Names: Compiled from the 2005 China 1% Population Census.
  • 20,000 Names: Collected from various public sources in China.
  • 58M Names: A massive dataset of 58,393,173 records from the Guangdong province government, used to evaluate the model against commercial services that also rely on big data.

Key Results

  • Superiority over Commercial Tools: On the 58M dataset, the proposed model outperformed popular tools like Namsor and Genderize.io by 9.70% to 20.08% in relative accuracy.
  • Ablation Success: Removing the Knowledge Distillation or the Multi-Task module led to a progressive decline in accuracy, proving that "character-awareness" is the key to its success.
  • Bias Reduction: The model significantly reduced the errorCoded metric (which includes both misclassifications and "unknown" results) compared to commercial APIs, which often fail to classify Chinese names entirely.

Discussion and Limitations

While our model reaches the upper bound of what is mathematically possible given phonetic input, a gap remains. Accuracy on Pinyin (approx. 71-73%) is still lower than the state-of-the-art for Hanzi (approx. 81%) due to the inherent ambiguity of the Pinyin system. Furthermore, naming conventions in China evolve over time; a model trained on 2005 census data may require adaptation for modern trends.

Conclusion

By moving beyond simple phonetics and leveraging the semantic depth of Hanzi through Knowledge Distillation, we provide a path toward more inclusive gender bias research. This "many-to-one" distillation approach could potentially be applied to other romanized languages, such as Japanese, where similar information loss occurs.

Summary updated

Abstract

Achieving gender equality is a pivotal factor in realizing the UN's Global Goals for Sustainable Development. Gender bias studies work towards this and rely on name-based gender inference tools to assign individual gender labels when gender information is unavailable. However, these tools often inaccurately predict gender for Chinese Pinyin names, leading to potential bias in such studies. With the growing participation of Chinese in international activities, this situation is becoming more severe. Specifically, current tools focus on pronunciation (Pinyin) information, neglecting the fact that the latent connections between Pinyin and Chinese characters (Hanzi) behind convey critical information. As a first effort, we formulate the Pinyin name-gender guessing problem and design a Multi-Task Learning Network assisted by Knowledge Distillation that enables the Pinyin embeddings in the model to possess semantic features of Chinese characters and to learn gender information from Chinese character names. Our open-sourced method surpasses commercial name-gender guessing tools by 9.70\% to 20.08\% relatively, and also outperforms the state-of-the-art algorithms.
BibTeX
@inproceedings{du2024misgendered,
  author = {Xiaocong Du and Haipeng Zhang},
  title = {{For the Misgendered Chinese in Gender Bias Research: Multi-Task Learning with Knowledge Distillation for Pinyin Name Gender Prediction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure