Leveraging Domain Knowledge for Inclusive and Bias-aware Humanitarian Response Entry Classification

Nicolò Tamagnone, Selim Fekih, Ximena Contla, Nayid Orozco, Navid RekabsazIJCAI 2023 · DOI

TL;DR

This paper introduces HumBERT, a novel language model specifically trained on humanitarian data, and a new architecture that better captures the hierarchical nature of humanitarian response frameworks. The work also presents HUMSETBIAS, a dataset for identifying and mitigating gender and country biases in humanitarian text classification, demonstrating significant bias reduction without performance loss.

What Problem Does This Paper Address?

In humanitarian crises, rapid and accurate analysis of incoming information is crucial for effective aid delivery. This often involves classifying text data according to established humanitarian frameworks. However, simply fine-tuning generic large language models (LLMs) for this task faces several challenges:

  • Complexity of the Framework: Humanitarian frameworks are often hierarchical and have intricate relationships between categories, which standard flat classification approaches struggle to capture.
  • Data Sparsity and Novelty: New crises often present unique situations with limited pre-existing data, making it difficult for models to perform well in zero-shot or few-shot settings.
  • Societal Biases: LLMs trained on general text can inherit and even amplify societal biases related to gender, geography, and other attributes. This is particularly problematic in humanitarian contexts, where inclusivity and impartiality are core principles (e.g., the Leave No One Behind - LNOB principle).

This paper aims to address these issues by developing an ethically-aware system for humanitarian data classification that is both effective and inclusive.

Methods

The authors propose a three-pronged approach:

  1. Novel Architecture: They introduce a combinatorial model that leverages shared Transformer layers from a base LLM for all categories, but then splits into task-specific sub-layers for different levels of the humanitarian framework (single-level vs. two-level tasks). This aims to better model the hierarchical structure and relationships within the framework, preventing "catastrophic forgetting" that can occur in multi-task learning.
  2. Domain-Specific LLM (HumBERT): To address data sparsity and domain specificity, they create HumBERT. This is a multilingual LLM fine-tuned on a large corpus (~2M documents) of humanitarian text data, including reports and news articles from sources like ReliefWeb and UNHCR Refworld. This fine-tuning allows HumBERT to capture the nuances and specialized vocabulary of the humanitarian domain.
  3. Bias Measurement and Mitigation:
    • HUMSETBIAS Dataset: A subset of the HUMSET dataset is created, specifically focusing on data points that might exhibit gender or country biases. This dataset includes targeted counterfactual samples, where sensitive words (e.g., gendered pronouns, country names) are systematically replaced to create altered versions of the original text.
  4. Bias Metrics: Two metrics are introduced: "Tag-Shift" to measure bias for specific tags and counterfactual transitions, and "Overall-Shift" to provide an aggregated measure of bias across all tags and transitions.
  5. Counterfactual Data Augmentation (CDA): The authors use the HUMSETBIAS dataset with counterfactual samples to augment the training data. This process aims to teach the model to be invariant to these sensitive attribute changes, thereby mitigating bias.

Datasets

  • HUMSET: Used for training and evaluating the classification models. This dataset contains humanitarian document entries classified according to a specific framework.
  • HUMSETBIAS: A newly introduced dataset, which is a subset of HUMSET focusing on gender and country biases. It includes original data points and targeted counterfactual samples.

Key Insights and Ideas

  • Hierarchical Architectures Matter: The proposed architecture, which explicitly models the hierarchical structure of the humanitarian framework, shows improved performance, especially for categories with many labels. This suggests that simply treating all labels as flat categories is suboptimal.
  • Domain-Specific LLMs are Crucial: Fine-tuning a general LLM on domain-specific data (HumBERT) leads to better performance, particularly in zero-shot settings, highlighting the value of adapting models to specialized language.
  • Bias is Pervasive and Measurable: The HUMSETBIAS dataset and the proposed bias metrics clearly demonstrate the presence of gender and country biases in LLMs applied to humanitarian text.
  • Counterfactual Data Augmentation is Effective: Using counterfactual data augmentation significantly reduces measured biases without negatively impacting classification performance, and in some cases, even slightly improving it. This is a promising direction for building more equitable AI systems.

Key Results

  • Zero-Shot Performance: HumBERT, when used with the proposed architecture, outperforms generic LLMs (m-BERT, XLM-R) in zero-shot classification across most categories, with notable improvements in F1-scores.
  • Full-Training Performance: The proposed architecture consistently shows higher performance, especially in F1-scores, compared to the baseline architecture across all LLMs. HumBERT, in particular, achieves the best performance across all categories.
  • Bias Reduction: The Tag-Shift and Overall-Shift metrics reveal significant biases in the models. After applying CDA, the biases are substantially reduced. For example, the Overall-Shift metric shows a significant decrease in bias for all LLMs after CDA.
  • Performance Preservation: Crucially, the bias mitigation techniques (CDA) did not lead to a drop in classification performance; in some cases, slight improvements were observed.

Limitations and Future Work

  • Bias Mitigation Completeness: While CDA significantly reduces bias, the paper acknowledges that models still contain bias. The HUMSETBIAS dataset does not cover all possible societal biases, and the authors encourage efforts to expand it to include attributes like specific needs groups, affected groups, and age categories.
  • Dataset Accuracy: The HUMSETBIAS dataset was created using rule-based mappings and underwent review, but minor shifts in results due to potential errors are acknowledged. Continuous improvement of the dataset is a goal.
  • Subpillar Category Performance: The models showed weaker performance on Subpillars 1D/2D categories. The authors attribute this to the complexity of the labels and limitations of the manual verbalizer used for zero-shot evaluation, suggesting further investigation in this area.
  • Generalizability: The effectiveness of the proposed methods on other humanitarian datasets or in different crisis contexts would be a valuable area for future research.

Summary updated

BibTeX
@inproceedings{tamagnone2023leveraging,
  author = {Nicolò Tamagnone and Selim Fekih and Ximena Contla and Nayid Orozco and Navid Rekabsaz},
  title = {{Leveraging Domain Knowledge for Inclusive and Bias-aware Humanitarian Response Entry Classification}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/690},
}
Figure