Fairness in Automated Essay Scoring: A Comparative Analysis of Algorithms on German Learner Essays from Secondary Education

Nils-Jonathan Schaller, Yuning Ding, Andrea Horbach, Jennifer Meyer, Thorben JansenNAACL 2024

TL;DR

This paper investigates the fairness of Automated Essay Scoring (AES) algorithms for German secondary school students, finding that while models trained on diverse data are generally fair, training on skewed subsets can lead to biased scores. The study highlights the critical need to include diverse student characteristics, including cognitive abilities, in training data to ensure equitable AES performance.

What Problem Does This Paper Address?

Ensuring educational equity is paramount, especially when leveraging technology for learning. Automated Essay Scoring (AES) systems offer a scalable way to provide students with feedback on their writing. However, a critical concern is whether these algorithms are "fair" – meaning they don't exhibit prejudice or favoritism towards certain student groups. Previous research has shown that AES can sometimes produce unfair scores, particularly for students from different demographic backgrounds. This paper extends this research by examining fairness in AES for German learner essays, going beyond demographic variables to also consider psychological characteristics like cognitive abilities. It also investigates how the composition of training data impacts both the accuracy and fairness of AES predictions.

Methods

The researchers employed a comparative analysis of three types of machine learning models:

  • Shallow Learning: A Support Vector Machine (SVM) classifier.
  • Deep Learning: A BERT-based classifier.
  • Large Language Models (LLMs): GPT-4 was used for generative tasks.

They evaluated these models on various annotation tasks within essays, including Introduction, Conclusion, Major Claim, Position, and Warrant. Fairness was assessed using metrics like Overall Score Accuracy (OSA), Overall Score Difference (OSD), and Conditional Score Difference (CSD), with a threshold of 0.1 for absolute values indicating potential unfairness. Performance was measured using Cohen's kappa to account for chance agreement.

A key aspect of their methodology was the experimental design:

  1. Full Model Evaluation: Models were trained on the complete DARIUS corpus and then evaluated for fairness across different demographic and cognitive subgroups.
  2. Subgroup Training Experiment: Models (excluding LLMs due to their zero-shot nature) were trained on specific subsets of the training data, focusing on students from the lowest and highest quartiles of cognitive ability (KFT scores), as well as a mixed subset. This was done to understand the impact of skewed training data on fairness.

Datasets

The primary dataset used in this study is the DARIUS corpus. This is a newly released corpus containing 4,589 annotated argumentative texts written by 1,839 German secondary school students. The essays cover topics related to energy and automotive issues. The corpus is annotated with various features, including content zone, major claim, position, and warrant. Demographic data (grade, gender, school type, profile, languages) and psychological data (cognitive ability via the KFT) were also collected for the students.

Key Insights and Ideas

  • Diversity is Key for Fairness: The study strongly suggests that training AES models on a diverse range of student characteristics is crucial for achieving fair scores.
  • Cognitive Ability Matters: Beyond demographic factors, psychological characteristics like cognitive ability (measured by KFT) significantly influence AES fairness.
  • Skewed Data Breeds Bias: Training models on data exclusively from high- or low-cognitive ability subgroups, or even just one demographic group, can lead to unfair outcomes for students not represented in that training subset.
  • LLMs vs. Traditional Models: While LLMs show promise, deep learning models (like BERT) often demonstrated robust performance and fairness in this specific German corpus. The performance of LLMs can be highly variable and prompt-dependent.
  • Annotation Quality Influences Fairness: The study notes that the inter-annotator agreement for certain annotation types (like Position and Warrant) was lower, which can mirror challenges in capturing nuanced argumentative elements and potentially impact model fairness.

Key Results

  • Full Model Fairness: When trained on the complete DARIUS corpus, all evaluated models (Shallow, Deep, and LLM) demonstrated fair scores across demographic and cognitive subgroups, with fairness metrics (difference in R²) consistently below the 0.1 threshold.
  • Performance Varies by Annotation Type: Deep learning models generally achieved higher kappa values (indicating better agreement with human annotations) for most annotation types, especially Introduction and Major Claim. Position and Warrant annotations showed lower performance across all models, reflecting challenges in capturing these nuanced elements.
  • Subgroup Training Impacts Fairness: In the experiment where models were trained on KFT subgroups:
  • Models trained on a single quartile (e.g., only high KFT students) showed reduced performance and potential unfairness when evaluated on students outside that quartile.
  • The "mixed" subset (representing 25% from each quartile) generally showed the lowest variance in performance and was more robust.
  • There were exceptions where models trained on a specific subgroup performed better on that subgroup, but this often came at the cost of fairness for other groups.

Limitations and Future Work

  • Corpus Size and Subgroup Representation: The study acknowledges that the DARIUS corpus has a small number of students in certain subgroups (e.g., specific family languages, Linguistics/Aesthetics profiles), which limits the robustness of conclusions for these groups.
  • Generalizability: The population of Schleswig-Holstein, Germany, may not fully represent the diversity of students in other regions or countries, potentially limiting the generalizability of the findings.
  • Future Work: The authors advocate for future AES systems to explicitly incorporate fairness goals into their training data collection and optimization functions, moving beyond retrospective analysis of model fairness. They suggest that future research should continue to explore the impact of diverse student characteristics, including psychological aspects, on AES performance and fairness.

Summary updated

BibTeX
@inproceedings{schaller2024fairness,
  author = {Nils-Jonathan Schaller and Yuning Ding and Andrea Horbach and Jennifer Meyer and Thorben Jansen},
  title = {{Fairness in Automated Essay Scoring: A Comparative Analysis of Algorithms on German Learner Essays from Secondary Education}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure