TL;DR
This paper introduces and evaluates a customized continuous delivery (CD) workflow implemented in a multi-customer software engineering project course. The authors demonstrate that integrating CD from the start significantly improves student understanding, engagement, and the quality of delivered software, bridging the gap between development and customer feedback.
What Problem Does This Paper Address?
The paper addresses the challenge of effectively teaching and implementing continuous delivery (CD) practices within an academic setting, specifically in a multi-customer project course. While CD is known to accelerate software releases and improve feedback loops in industry, its complexity can be a barrier for students new to software development. The authors aim to bridge this gap by creating a practical, integrated CD workflow and evaluating its impact on student learning, adoption, and perceived benefits. They also highlight the difficulty of introducing these concepts late in a course, as experienced in a previous iteration.
Methods
The core of the paper is the description of a customized continuous delivery workflow built upon the principles of Humble and Farley's deployment pipeline. This workflow integrates several tools: JIRA for issue tracking, Stash (with Git) for version control, Bamboo for continuous integration, and HockeyApp for software delivery.
Key aspects of their approach include:
- Integrated Deployment Pipeline: A multi-stage process where builds move from integration through testing to production, with automated quality checks at each stage.
- Simplified Gitflow: A modified version of the gitflow branching model is used to manage feature development and releases, distinguishing between internal and external releases.
- Release Manager Role: Each project team designates a "release manager" responsible for CD activities within their team. A "release coordinator" (a teaching assistant) oversees all release managers.
- Teaching Methods: A multi-pronged approach is used to transfer knowledge:
- Cross-Project Release Management Team: A dedicated team of release managers meets regularly to share knowledge, synchronize understanding, and resolve issues.
- Lectures: A course-wide presentation introduces the core concepts of Git, Bamboo, and HockeyApp, explaining the workflow and its benefits.
- Tutorials: Step-by-step manuals and video tutorials are provided for practical guidance on using the tools.
- Early Integration: The CD workflow and tools are introduced from the very beginning of the project course, unlike previous attempts where they were introduced later.
Datasets
Not discussed in the paper. The evaluation relies on data collected from student participants in the project course through online questionnaires and personal interviews.
Key Insights and Ideas
- Early Integration is Crucial: Introducing CD from the start, rather than later, significantly improves student adoption and understanding. This allows students to build foundational knowledge and practice throughout the project.
- Dedicated Roles Enhance Adoption: The creation of "release managers" within teams and a "release coordinator" for the entire course fostered ownership and facilitated the dissemination of CD knowledge.
- Practical Tools Simplify Complexity: Using well-integrated, university-friendly tools (JIRA, Stash, Bamboo, HockeyApp) made the complex CD concepts more accessible and manageable for students.
- Bridging the Dev-Customer Gap: The CD workflow, with its focus on frequent, easy releases, effectively shortened feedback cycles between student developers and their industry customers, leading to earlier validation of requirements and improved communication.
- Motivation Through Visibility: The use of a "wallboard" visualizing the build and release status motivated students to maintain a potentially releasable state of their software.
Key Results
- High Understanding: Over 90% of students felt they could explain the concepts of version control, continuous integration, and continuous delivery to a friend after the course.
- Significant Usage: 45% of students reported using continuous integration and continuous delivery in their daily routine. Version control tasks were used most frequently.
- Perceived Benefits: The vast majority of students (around 80%) saw clear benefits in using version control, continuous integration, and continuous delivery.
- Future Application: 70% of participants indicated they would likely apply these concepts in future projects.
- Increased Releases: The number of delivered builds to customers increased by over 400% in 2013 compared to 2012, with a significantly higher percentage of successful builds.
- Branching Adoption: The introduction of a branching model led to a substantial increase in the number of branches used.
Limitations and Future Work
- Selection Bias: The authors acknowledge that not all students participated in the questionnaire, potentially introducing a selection bias. However, they mitigated this by ensuring at least three responses per team and finding consistent results across teams.
- Generalizability to Industry: The authors recognize that the university setting differs from industry. While they believe their project-based organization closely mirrors industry practices, they plan to evaluate the applicability of their findings in an industry project in 2014.
- Structured Feedback Usage: While the delivery server offered structured feedback mechanisms, students often relied more on personal interactions (customer meetings, phone calls) for feedback, suggesting that structured tools are more beneficial when a large number of test users are available or personal meetings are not feasible.
- Future Work: The authors plan to evaluate their approach in an industry project to assess the generalizability of their findings.
Summary updated