"You might think about slightly revising the title": Identifying Hedges in Peer-tutoring Interactions

Yann Raphalen, Chloé Clavel, Justine CassellACL 2022 · DOI

Introduction: The Social Nuance of Tutoring

In intelligent tutoring systems, the effectiveness of instruction is often tied to the quality of the social bond, or "rapport," between the tutor and the student. A critical linguistic tool for managing this rapport is hedging—the use of softeners like "I think" or "kind of" to mitigate the directness of feedback or instructions. By reducing "face threat," hedges help prevent students from feeling embarrassed by mistakes, thereby maintaining a supportive learning environment.

While hedge detection is well-established for formal academic writing, it remains a challenge in casual, face-to-face conversation. Teenagers, in particular, use hedges in highly ambiguous ways. Words like "just" or "would" can serve as social softeners or simple descriptors depending on context. This research introduces a hybrid machine learning framework designed to automatically detect these subtle social signals in peer-tutoring interactions.

Methodology: A Hybrid Linguistic Approach

The researchers moved away from purely "black-box" deep learning models, opting instead for a hybrid approach that prioritizes interpretability and linguistic domain knowledge. They identified four primary classes of hedges relevant to tutoring:

  1. Subjectivizers: Phrases like "I think" that frame a statement as an opinion.
  2. Apologizers: Words like "Sorry" used to soften corrections.
  3. Propositional Hedges: Qualifiers like "sort of" or "actually" that reduce certainty.
  4. Extenders: Vague categories like "or something" or "and stuff."
Examples of the four hedge categories identified in the study.

The framework combines a rule-based foundation—using regular expressions to identify these classes—with a massive array of engineered features. These include lexical (unigrams/bigrams), syntactic (POS n-grams), psychological (LIWC categories), multimodal (gaze, smiles), and contextual (tutoring moves) data.

Dataset and Experimental Setup

The study utilized a specialized multimodal peer-tutoring dataset featuring 24 American teenagers. The data comprises nearly 60 hours of face-to-face interaction, segmented into approximately 24,000 clauses. A significant challenge in this dataset is the class imbalance: over 90% of utterances are not hedged, making the detection of the remaining 10% a "needle in a haystack" problem.

Results: Knowledge-Driven Features vs. Deep Learning

The authors compared several architectures, including LightGBM (Gradient Boosting), Multi-layer Perceptrons (MLP), LSTMs, and fine-tuned BERT. The evaluation focused on a 4-class task (including non-hedged) and a more difficult 3-class task (focusing only on the rare hedge categories).

Performance comparison (F1-score) across different model architectures and feature sets.

The results demonstrated that the "Knowledge-Driven" LightGBM model significantly outperformed the deep learning heavyweights. Specifically, LightGBM achieved an F1-score of 79.0 for the minority hedge classes, compared to 70.6 for BERT and 64.5 for a previous deep learning baseline. This suggests that for niche social signals in specific domains like teen tutoring, explicit linguistic features and rule-based synergy are more effective than the generalized representations of massive language models.

Key Insights and Discussion

Analysis using SHAP (Shapley Additive Explanations) revealed several key linguistic patterns:

  • Lexical Dominance: Hedges are primarily a lexical phenomenon; specific words like "just," "would," and "sorry" carry more weight than complex syntax.
  • The "Oh" Factor: Interjections like "oh" and "ah" are powerful predictors for Apologizers (e.g., "Oh sorry, I meant twelve").
  • Clause Length: Longer clauses often contain Subjectivizers, while shorter ones tend to be more direct.

Conclusion and Future Work

While the current model is highly effective at the clause level, future work aims to incorporate prosodic features (pitch and tone) and broader conversational context. The ultimate goal is to move from detection to generation, enabling tutoring agents to produce appropriately polite and supportive responses in real-time to maximize student rapport and learning outcomes.

Summary updated

BibTeX
@inproceedings{raphalen2022you,
  author = {Yann Raphalen and Chloé Clavel and Justine Cassell},
  title = {{"You might think about slightly revising the title": Identifying Hedges in Peer-tutoring Interactions}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.acl-long.153},
}
Figure