Introduction
Intelligent Tutoring Systems (ITSs) have proven highly effective at providing personalized, step-by-step guidance to students. However, these systems often operate as "black boxes" for the teachers overseeing the classroom. While an ITS generates vast amounts of data regarding student knowledge, affect, and "wheel-spinning" (persistent struggling without progress), this information is rarely presented in a way that is easy for K-12 teachers to digest or act upon. Researchers at Carnegie Mellon University are addressing this gap by developing a teacher dashboard designed to transform high-resolution ITS data into actionable classroom instruction.
Methodology: User-Centered Design
The research team employed a multi-stage User-Centered Design (UCD) approach to ensure the dashboard met the practical needs of middle school teachers. This process began with 11.5 hours of contextual inquiry, including interviews and observations with six teachers to understand their current data usage.
To explore potential features, the researchers used a technique called "Speed Dating with Storyboards." They presented 22 different scenarios to teachers to gauge interest in features like automated student grouping or alerts for "gaming the system."
Following these initial phases, the team iterated from medium-fidelity mockups to a high-fidelity dashboard built in Tableau. This prototype was then evaluated in a controlled classroom study where teachers used real data from their own students to plan lessons.
Key Insights and Dashboard Design
The study revealed several critical requirements for educational dashboards in "blended" learning environments:
- The Actionability of Errors: Teachers were less concerned with simple completion metrics and more interested in why students were failing. In algebra, identifying specific misconceptions allows a teacher to immediately pivot to a targeted "mini-lesson."
- The Trust vs. Summary Trade-off: While teachers appreciated high-level summaries, they expressed a need to "drill down" into raw student steps to verify the system's conclusions.
- Teacher Agency: The dashboard is viewed as a support tool for teacher expertise, not an automated replacement for instructional decision-making.
The researchers identified two primary use cases: "on-the-spot" real-time triage during computer lab sessions, and "next-day" reflective planning used outside of class to adjust future instruction.
Results and Future Directions
Preliminary qualitative findings indicate that ITS-specific metrics, such as learning curves and wheel-spinning indicators, provide insights that traditional observation cannot capture. Access to the dashboard appeared to shift instructional focus toward common misconceptions identified by the software rather than general progress.
While the quantitative impact on student learning gains is still under analysis, the study highlights the necessity of specialized UI designs for different teaching scenarios. Future work will focus on fully integrating the dashboard into the live CTAT/Tutorshop backend to provide real-time analytics and optimizing the interface for mobile devices to support active classroom management.
Summary updated