More Insightful Feedback for Tutoring: Enhancing Generation Mechanisms and Automatic Evaluation

Wencke Liermann, Jin-Xia Huang, Yohan Lee, Kong Joo LeeEMNLP 2024 · OpenReview

Abstract

Incorrect student answers can become valuable learning opportunities, provided that the student understands where they went wrong and why. To this end, rather than being given the correct answer, students should receive elaborated feedback on how to correct a mistake on their own. Highlighting the complex demands that the generation of such feedback places on a model’s input utilization abilities, we propose two extensions to the training pipeline. Firstly, we employ a KL regularization term between a standard and enriched input format to achieve more targeted input representations. Secondly, we add a preference optimization step to encourage student answer-adaptive feedback generation. The effectiveness of those extensions is underlined by a significant increase in model performance of 3.3 METEOR points. We go beyond traditional surface form-based metrics to assess two important dimensions of feedback quality, i.e., faithfulness and informativeness. Hereby, we are the first to propose an automatic metric measuring the degree to which feedback divulges the correct answer, that we call Informativeness Index I2. We verify in how far each metric captures feedback quality.

Introduction

In automated tutoring, the most effective learning occurs when students are guided to discover their own errors. While simple verification (telling a student they are wrong) or direct correction (providing the right answer) are common, educational research favors elaborated feedback. This involves providing hints or explanations that encourage self-reflection and deeper engagement with the material.

The spectrum of feedback types, from simple verification to detailed correction.

Generating such feedback is a complex task for Large Language Models (LLMs). A model must simultaneously synthesize the source text, the specific question, and the student's unique incorrect answer to provide a response that is faithful to the material, adaptive to the student's logic, and pedagogically sound—meaning it should not prematurely reveal the correct answer.

The ReCTify Framework

Existing models often struggle with "input utilization," frequently ignoring the source text or simply echoing the student's error. To address these challenges, we developed ReCTify, a framework that enhances the training pipeline of a T5-base model through two primary mechanisms:

1. Key Sentence-Aided KL Regularization

To help the model identify relevant information within long reading passages, we utilize "key sentence" annotations. During training, the model processes two versions of the input: a standard version and an "enriched" version where key sentences are highlighted. A Kullback-Leibler (KL) regularization term is used to align the model's output distribution for the standard input with that of the enriched input. This effectively trains the model to implicitly prioritize key sentences during inference without requiring explicit highlights.

2. Answer-Adaptive Direct Preference Optimization (DPO)

To prevent redundant feedback—such as simply negating a student's wrong statement—we employ Direct Preference Optimization. Using a Natural Language Inference (NLI) model, we score generated feedback based on its entailment with the student's incorrect answer. The model is trained to prefer responses with low entailment, which ensures the feedback provides new, insightful information rather than just echoing the error.

The ReCTify training pipeline featuring Key Sentence-Aided KL Regularization and Answer-Adaptive DPO.

Evaluating Pedagogical Quality

Traditional NLP metrics like BLEU and ROUGE often fail to capture pedagogical value, sometimes even favoring smaller models that match reference lengths over more capable models like GPT-4. To solve this, we introduced two new metrics:

  • Informativeness Index ($I^2$): Measures how much a piece of feedback supports the correct answer relative to a gold reference, using a model trained on the MultiRC task.
  • Faithfulness ($F$): Ensures the feedback remains grounded in the source material and does not introduce contradictions.

Results and Insights

Our full ReCTify model achieved a significant 3.3-point boost in METEOR scores over the baseline. DPO proved to be a critical component, contributing a 2.7-point increase by successfully penalizing redundant feedback.

Furthermore, our proposed $I^2$ metric demonstrated a Spearman correlation of 0.52 with human judgments, far outperforming traditional metrics like BLEU (0.34) or METEOR (0.20). In blind evaluations, humans preferred GPT-4 feedback 50% of the time, followed by GPT-3.5 (38%), highlighting that while our T5-based improvements are substantial, larger models still hold an advantage in nuanced pedagogical reasoning.

Conclusion

By focusing on input utilization and answer-adaptivity, ReCTify moves automated tutoring toward more meaningful student interactions. The introduction of the Informativeness Index provides a more human-aligned way to measure the success of these systems, ensuring that feedback is not just linguistically accurate, but educationally effective.

Summary updated

Abstract

Incorrect student answers can become valuable learning opportunities, provided that the student understands where they went wrong and why. To this end, rather than being given the correct answer, students should receive elaborated feedback on how to correct a mistake on their own. Highlighting the complex demands that the generation of such feedback places on a model’s input utilization abilities, we propose two extensions to the training pipeline. Firstly, we employ a KL regularization term between a standard and enriched input format to achieve more targeted input representations. Secondly, we add a preference optimization step to encourage student answer-adaptive feedback generation. The effectiveness of those extensions is underlined by a significant increase in model performance of 3.3 METEOR points. We go beyond traditional surface form-based metrics to assess two important dimensions of feedback quality, i.e., faithfulness and informativeness. Hereby, we are the first to propose an automatic metric measuring the degree to which feedback divulges the correct answer, that we call Informativeness Index I2. We verify in how far each metric captures feedback quality.
BibTeX
@inproceedings{liermann2024more,
  author = {Wencke Liermann and Jin-Xia Huang and Yohan Lee and Kong Joo Lee},
  title = {{More Insightful Feedback for Tutoring: Enhancing Generation Mechanisms and Automatic Evaluation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=GDSxzyZDlg},
}
Figure