TL;DR
This paper investigates how adding specific features to a spoken dialogue system's understanding of the user's state impacts its decision-making. By using reinforcement learning on a physics tutoring dialogue corpus, they found that features like "concept repetition" and "frustration" significantly alter the tutor's actions, suggesting that richer user state representations lead to more adaptive tutoring.
What Problem Does This Paper Address?
Designing effective spoken dialogue systems, especially in educational contexts, involves two major challenges: deciding the best system action at any given moment and accurately representing the user's current state. While Reinforcement Learning (RL) has shown promise in tackling the former, there's been less empirical work on the latter – specifically, identifying which features are most crucial for representing the user's state. This paper aims to fill that gap by systematically evaluating the impact of adding several new features to a baseline user state representation in a spoken dialogue tutoring system. The goal is to understand which features are most beneficial for improving the tutor's ability to adapt its questions and maximize student learning.
Methods
The authors frame the dialogue management problem as a Markov Decision Process (MDP). They use an existing RL infrastructure that allows for policy iteration to learn optimal actions based on states and rewards. The core of their methodology involves:
- Establishing a Baseline: They first define a baseline state space (initially just "correctness" of the student's answer) and a set of tutor actions (asking different types of questions: Simple Answer Question (SAQ), Complex Answer Question (CAQ), or a mix). The reward function was based on student learning gain.
- Adding Features Incrementally: They then systematically add new features to the baseline state space one by one. These features include:
- Certainty: How confident the student is in their answer.
- Concept Repetition: Whether a concept has been previously covered and needs re-explanation.
- Frustration: The student's emotional state.
- Percent Correctness: The student's overall performance on the current problem.
- Evaluating Policy Changes: For each added feature, they re-run the MDP policy iteration and compare the resulting policies and state-value functions to the baseline. They use three metrics to quantify the impact:
- # of Diff's: The raw count of states whose optimal action changed.
- % Policy Change: A weighted measure of policy changes, accounting for state occurrence frequency.
- Expected Cumulative Reward (E.C.R.): A measure of the overall utility of the new policy.
- Testing Different Actions: They also re-evaluate the impact of these features using a different set of tutor actions: providing different types of feedback (Simple, Complex, or Mix).
Datasets
The study uses an annotated corpus of 20 human-computer spoken dialogue tutoring sessions. These sessions involved students interacting with the ITSPOKE system, which uses the text-based Why2-ATLAS system for physics tutoring. Each session covered 5 college-level physics problems, totaling approximately 100 dialogues. The corpus was pre-annotated for tutor moves (dialogue acts) and the features used in this study (certainty, frustration) were also manually annotated.
Key Insights and Ideas
- Richer State, Better Decisions: The core idea is that a more detailed representation of the user's state leads to more nuanced and potentially more effective system actions.
- Concept Repetition is Key: This feature had the most significant impact on the tutor's policy, suggesting that knowing when a student is struggling with a repeated concept is crucial for adapting the tutoring strategy.
- Frustration Matters, Especially for Questions: Frustration significantly altered tutor actions, particularly when deciding what type of question to ask.
- Certainty's Impact Varies: While certainty did change policies, its impact was less pronounced than concept repetition or frustration.
- Percent Correctness is Less Influential: Adding "Percent Correctness" as a feature resulted in minimal policy changes, suggesting it might be less critical than other indicators of immediate student understanding or engagement.
- Generalizability Across Actions: The relative importance of features (Concept Repetition > Frustration > Percent Correctness) held true for both question-based and feedback-based tutor actions, though the magnitude of change differed.
Key Results
- Concept Repetition: Led to the largest policy changes (10 differences out of 12 states) and the highest % Policy Change (80.2%) and E.C.R. (39.52) when using question actions.
- Frustration: Resulted in 8 policy differences and a significant % Policy Change (66.4%) and E.C.R. (31.30).
- Percent Correctness: Showed the least impact, with only 4 policy differences, a % Policy Change of 44.3%, and an E.C.R. of 28.17.
- When using feedback actions, Concept Repetition still showed the greatest impact, while Frustration and Percent Correctness had lower % Policy Changes, indicating they are less critical for feedback decisions compared to question decisions. However, the E.C.R. for Concept Repetition with feedback was higher than with questions.
Limitations and Future Work
- Manual Annotation: Certainty and frustration features were manually annotated. The authors acknowledge the need for automated detection algorithms for these features, noting that previous work has shown promising accuracy for automated certainty detection.
- Corpus Size: While the study used a reasonable corpus, they suggest that a larger corpus would allow for more complex states and tutor actions to be explored.
- Future Metrics: They plan to investigate other metrics like entropy and confidence bounds to better indicate feature usefulness.
- Automated Feature Detection: A key future direction is to investigate how well automated certainty and frustration detection algorithms impact the % Policy Change, as manual annotation is costly.
- More Complex Actions: Exploring more complex tutor actions like "Hints" and "Restatements" is also a future goal.
Summary updated