TL;DR
This paper introduces ClassroomWiki, a novel system that leverages multiagent technology to enhance collaborative writing in educational settings. By tracking student activities, modeling their contributions, and intelligently forming groups, ClassroomWiki aims to improve learning outcomes, foster better collaboration, and combat issues like free-riding.
What Problem Does This Paper Address?
Collaborative writing using wikis is a popular pedagogical tool, but it faces significant challenges. Firstly, wikis often lack mechanisms for effective group formation, leading to suboptimal collaboration dynamics. Secondly, detecting and addressing "free-riding" – where students benefit without contributing – is difficult, which can demotivate diligent students and reduce overall participation. Existing systems struggle to accurately assess individual contributions and provide timely, targeted interventions to struggling students.
Methods
The core of ClassroomWiki is its multiagent system, inspired by the Multiagent Human Coalition Formation (MHCF) framework. Each student is assigned an intelligent agent responsible for:
- Tracking: Monitoring student interactions within the wiki environment (e.g., words added/deleted, forum posts, revisions).
- Modeling: Building a student model based on tracked activities, encompassing their contribution (cni,t), collaborative effort (coi,t), and teacher/peer evaluations (evi,t).
- Group Formation: Using a Bayesian Network to probabilistically map student models to group performance. Agents then negotiate to form student groups that are predicted to maximize both current-task performance and future learning.
The system employs Natural Language Processing (NLP) to distinguish between trivial edits (like spelling corrections) and content-related changes, leading to more nuanced student models. The Bayesian Network is cooperatively built and updated by the agents, allowing it to learn and adapt group compositions over time.
Datasets
The paper describes deployments in three university-level courses and one community outreach activity. Specific courses mentioned are:
- HIST 202 (America after 1877)
- CSCE 475 (Multiagent Systems)
- ENGL 180 (Intro. to Literature)
- GEM (General Education Mentoring)
- WMNS101 (Women's Studies)
For the HIST 202 and CSCE 475 deployments, a control-treatment protocol was used, where students were divided into groups based on prior assessment scores to ensure similar starting points.
Key Insights and Ideas
- Agent-based modeling is key: By assigning an agent to each student, the system can continuously and granularly track individual contributions, moving beyond simple participation metrics.
- Bayesian Networks for probabilistic group formation: The use of Bayesian Networks allows the system to model the complex, probabilistic relationships between student attributes, group composition, and learning outcomes, enabling more informed group assignments.
- Adaptive group composition: The system can learn and adapt group compositions over time. In an introductory course (HIST 202), heterogeneous groups were found to be beneficial, while in an advanced course (CSCE 475), homogeneous groups emerged as more effective.
- Proactive intervention: Detailed tracking data allows instructors to identify potential free-riders or students struggling with collaboration early on, enabling timely scaffolding and support.
Key Results
- Improved Student Performance: Treatment groups (using ClassroomWiki's MHCF) showed statistically significant improvements in scores compared to control groups in HIST 202 (p < 0.05).
- Enhanced Collaboration: Treatment groups exhibited more collaborative interactions, measured by higher revision counts and forum activity, in both HIST 202 and CSCE 475.
- Alleviation of Free-Riding: The agent-based tracking successfully identified free-riders in the deployments, allowing instructors to intervene proactively.
- Accurate Student Modeling: Correlations between students' final exam scores and their ClassroomWiki assignment scores (0.75 for HIST 202, 0.67 for CSCE 475) suggest that the student models closely represented the quality of their contributions.
- Positive User Feedback: Students generally found ClassroomWiki useful, with some rating it higher than existing platforms like Blackboard's Wiki.
Limitations and Future Work
- Limited Learning Opportunity in Single-Session Formation: In the initial HIST 202 deployment, group formation occurred only once, limiting the Bayesian Network's ability to learn from iterative interactions. Future work could involve more frequent group formation sessions.
- Need for Further Validation: While promising, the results are based on a limited number of deployments. The authors plan to extend the user base and continue gathering data.
- Future Enhancements: The authors propose allowing students to evaluate each other's contributions and enabling teachers to annotate wikis to further enrich the tracking and modeling capabilities.
Summary updated