A Study Report in the Web Technologies Course: What Makes Feedback Effective for Project-based Learning?

Alaa Jaber, Kimberly LeChasseur, Khairul Mottakin, Zheng Song2024 · DOI

TL;DR

This study investigates how different types of instructor feedback—high-level "task" feedback versus detailed "process" feedback—impact student success in Project-Based Learning (PjBL). The researchers found that while detailed, process-oriented feedback is more demanding for instructors, it significantly improves students' technical outcomes and overall project quality without compromising their sense of ownership.

What Problem Does This Paper Address?

Project-Based Learning (PjBL) is a popular pedagogical strategy where students learn by tackling open-ended, real-world challenges. The goal is to foster "ownership" and "authenticity," but this creates a dilemma for instructors: how much feedback is too much?

In traditional teaching, "process-level" feedback (explaining how to do something or how to fix an error) is generally seen as superior to "task-level" feedback (simply stating what was done correctly or incorrectly). However, some educators worry that in a PjBL environment, providing too much detail might make students feel over-supervised, stifling their creativity and independence. This paper seeks to determine which feedback strategy actually works best in a technical, high-level computer science setting.

Methods

The researchers conducted a case study within a Web Technologies course involving 56 undergraduates (mostly seniors) divided into 15 project teams. The teams spent a semester building functional websites intended to benefit their university community.

The study employed a controlled comparison of two feedback strategies:

  1. Task-level (High-level) Feedback: Provided by one Teaching Assistant (TA-a), this focused on whether the submission met requirements (e.g., "The database schema is complete") without offering specific suggestions for improvement.
  2. Process-level (Low-level) Feedback: Provided by a second Teaching Assistant (TA-b), this included specific technical suggestions and strategies for error detection (e.g., "Use Memcached for caching" or "Implement Ajax to prevent page refreshes").

To ensure an unbiased evaluation, three external industry experts (web developers with 3+ years of experience) graded the final projects. These evaluators were "blind" to which group received which type of feedback.

Datasets

As this is an educational study rather than a machine learning paper, the "dataset" consists of the student cohort and their project outputs:

  • Participants: 56 undergraduate students.
  • Projects: 15 unique web applications, including interactive campus maps, e-commerce platforms, and second-hand book exchanges.
  • Evaluation Metrics: Projects were scored on a 25-point scale across five categories: Novelty, Design, Technical Depth, Presentation, and Q&A.

Key Insights and Ideas

  • The Technical Edge: The most significant performance gap occurred in "Technical Soundness." Students receiving process-level feedback were pushed to implement professional-grade features (like database indexing and automated testing) that they likely wouldn't have discovered on their own.
  • The Autonomy Paradox: While task-level feedback was intended to preserve student autonomy, it sometimes left students frustrated. Without specific guidance, some students struggled to understand why a mistake was a mistake, whereas process-level feedback provided a clearer roadmap for learning.
  • The Expertise Requirement: Providing effective process-level feedback is hard. The study noted that the TA providing low-level feedback needed significant industry experience (4 years as a web developer) to offer the kind of "pro-tips" that actually moved the needle for students.
  • The "Halo Effect" in Grading: Interestingly, students who received better technical feedback also scored higher on their presentations, even though the TAs provided no help with public speaking. The researchers suggest that evaluators' positive impressions of the technical work "bled over" into their assessment of the presentation quality.

Key Results

The quantitative results clearly favored the more detailed feedback approach:

  • Total Scores: The process-level group averaged 22.9/25, while the task-level group averaged 20.3/25.
  • Technical Depth: The process-level group saw a 0.7-point lead (on a 5-point scale) over the task-level group, the largest single-category difference.
  • Qualitative Feedback: Interviews revealed that students overwhelmingly preferred in-person feedback over written comments, as it allowed for immediate clarification and dialogue.

Limitations and Future Work

The authors acknowledge several limitations:

  • Sample Size: The study was limited to a single course and a relatively small number of projects (15).
  • TA Variables: The two TAs had different backgrounds (one was a former lecturer, one a developer), which may have influenced the quality of feedback beyond just the "level" of detail.
  • Workload: Process-level feedback is significantly more time-consuming for instructors, a factor that needs to be balanced against the learning gains.

Future Work: The researchers plan to conduct more extensive experiments, including entrance questionnaires to account for students' prior knowledge and more precise tracking of the hours TAs spend providing feedback to measure the "cost-to-benefit" ratio of these strategies.

Summary updated

Abstract

Abstract In this paper, we report our experience of applying project-based learning (PjBL) in a web technology course, with a focus on how different feedback tactics affect students' learning outcomes. Giving high-quality feedback can have an immediate impact on students' learning progress. Although process-level feedback is considered more effective than task-level feedback in traditional pedagogies, it is still unclear whether this conclusion can be exacerbated or mitigated when applied to PjBL, as PjBL emphasizes students' ownership and authenticity. We divided students into two groups and applied different feedback strategies. We report findings and insights to help instructors develop feedback policies and practices that are effective and likely to benefit students. Our results suggest that giving low-level feedback still improves students' learning outcomes in the context of PjBL, especially for technical tasks.
BibTeX
@misc{jaber2024study,
  author = {Alaa Jaber and Kimberly LeChasseur and Khairul Mottakin and Zheng Song},
  title = {{A Study Report in the Web Technologies Course: What Makes Feedback Effective for Project-based Learning?}},
  year = {2024},
  doi = {10.18260/1-2--42508},
}

Loading related content...

Figure