THE ROLE OF CYBER COMPETITIONS IN CYBER DEFENSE EDUCATION: A CASE STUDY OF NATIONAL CYBER LEAGUE (NCL) PARTICIPATION

Ping Wang, Robert H. Morris, Hubert Cruze, J Blair, A Hall, E Sobiesk, K Cantaneo, A Conklin, F Jiwani, A Joyce, S Day, N Evans, M Kapoor, S Hua, D Anastasiu, M Kornegay, M Arafin, K Kornegay, La Fleur, C Hoffman, B Gibson, C Buchler, N, M Leijon, P Gudmundsson, P Staaf, C Christersson, F Orji, K Oyibo, J Greer, J Vassileva, J Rursch, D Jacobson, M Dawson, K Williams, R SbeitIssues 2022 · DOI

TL;DR

This paper evaluates the National Cyber League (NCL) as a pedagogical tool for bridging the gap between theoretical cybersecurity education and the hands-on skills required by the industry. By mapping NCL activities to national standards (NCAE-CDE), the authors demonstrate that these competitions not only boost student motivation but also correlate with higher academic success in related coursework.

What Problem Does This Paper Address?

The cybersecurity industry is facing a massive workforce crisis, with a global shortage of approximately 2.7 million professionals. While universities are churning out graduates, there is often a disconnect between classroom theory and the "on-the-ground" technical skills needed to defend critical infrastructure.

Furthermore, for institutions seeking the prestigious National Centers of Academic Excellence in Cyber Defense (NCAE-CDE) designation—sponsored by the NSA and DHS—student participation in cyber competitions is a mandatory requirement for quality assurance. The authors address the need for a formal framework to assess whether these competitions actually improve learning outcomes or if they are just "fun" extracurriculars.

Methods

The authors utilize the Challenge Based Learning (CBL) framework to analyze the National Cyber League. CBL is an active learning pedagogy divided into three phases: Engage, Investigate, and Act.

The study’s methodology involves:

  1. Mapping: Aligning NCL’s nine technical domains (e.g., Log Analysis, Cryptography) with the NICE Cybersecurity Workforce Framework and the specific Knowledge Units (KUs) required for NCAE-CDE designation.
  2. Case Study: A longitudinal observation of undergraduate students at a private university over eight seasons (Fall 2018 to Spring 2022).
  3. Integration: Embedding NCL participation as an optional, incentivized component in three core courses: Networks & Data Communication, Computer Network Security, and Ethical Hacking.

Datasets

The paper relies on a primary dataset collected from a private non-profit university in the northeastern U.S.

  • Participants: Undergraduate students enrolled in BS Cybersecurity programs.
  • Timeframe: Eight consecutive NCL seasons.
  • Metrics: The authors tracked "NCL Domain Completion" (the percentage of challenges a student finished) and compared the "Participant Course Success Rate" against the "Overall Course Success Rate" for the entire class.

Key Insights and Ideas

  • Virtualization as an Equalizer: Unlike many high-stakes, in-person competitions that require expensive hardware labs and travel budgets, NCL’s cloud-based, asynchronous model ($35 registration) makes it highly scalable and accessible for diverse student populations.
  • The "Aha" Moment of Mapping: The most significant contribution is the detailed mapping in Table 1. By showing exactly how a "Capture the Flag" (CTF) challenge in "Network Traffic Analysis" satisfies specific federal Knowledge Units (like BNW or NDF), the authors provide a blueprint for faculty to use competition data as formal evidence for accreditation.
  • Intrinsic Motivation through Gamification: Qualitative feedback showed that students viewed the "hardness" of the challenges as a positive. The competitive element stimulated a desire to "enroll again," suggesting that gamification helps sustain interest in a notoriously difficult field.
  • Correlation with Success: The data suggests that students who engage deeply with NCL (higher completion rates) tend to perform better in their formal coursework than the average student.

Key Results

  • Academic Outperformance: In almost every season studied, students who participated in NCL had a higher course success rate (defined as achieving 70% or higher) than the class average. For example, in Spring 2021, NCL participants had a 96.8% success rate compared to the 91.2% overall class average.
  • Increasing Engagement: There was a general upward trend in the average NCL domain completion rate over the four-year period, moving from 66.5% in 2018 to nearly 80% by 2022.
  • Direct Curricular Alignment: The authors successfully demonstrated that NCL domains overlap significantly with learning outcomes in Ethical Hacking and Network Security, such as identifying OSI model components and performing penetration testing.

Limitations and Future Work

The authors are transparent about the study's preliminary nature:

  • Small Sample Size: The number of voluntary participants per season was relatively low (ranging from 1 to 8 students), which limits the statistical power of the correlations.
  • Single Institution: The data comes from one private university, so the results may not be immediately generalizable to larger public institutions or different demographics.
  • Future Directions: The authors suggest future research should explore the impact of NCL on specific "soft skills" like critical thinking and problem-solving. They also aim to conduct comparative studies between NCL and other competition formats (like the CCDC) to see which pedagogical approach yields the best workforce readiness.

Summary updated

Abstract

This research paper explores the value of cybersecurity competitions in cyber defense education and its impact on the cybersecurity industry and workforce development. Competitions are considered active and challenge based learning that can be used as effective pedagogies to improve student interest, motivation, and problem solving in education. For cyber defense education quality assurance, student participation in cyber competitions is required for the National Centers of Academic Excellence in Cyber Defense Education (NCAE-CDE) designation by the US National Security Agency and Department of Homeland Security (NSA/DHS). This research is based on the Challenge Based Learning (CBL) framework and explores the pedagogical benefits of cybersecurity competitions through the case study of the National Cyber League (NCL) competition. The case study in this research focuses on mapping the features and knowledge and skill domains of the NCL competition to relevant NCAE-CDE program criteria and knowledge units and presents sample data on students' participation and performance from a NCAE-CDE designated institution. This study also analyzes the quantitative data and qualitative observations and reflections on student participation in the NCL competition and their performance in cyber defense educational programs.
BibTeX
@inproceedings{wang2022role,
  author = {Ping Wang and Robert H. Morris and Hubert Cruze and J Blair and A Hall and E Sobiesk and K Cantaneo and A Conklin and F Jiwani and A Joyce and S Day and N Evans and M Kapoor and S Hua and D Anastasiu and M Kornegay and M Arafin and K Kornegay and La Fleur and C Hoffman and B Gibson and C Buchler and N and M Leijon and P Gudmundsson and P Staaf and C Christersson and F Orji and K Oyibo and J Greer and J Vassileva and J Rursch and D Jacobson and P Wang and M Dawson and K Williams and P Wang and R Sbeit},
  title = {{THE ROLE OF CYBER COMPETITIONS IN CYBER DEFENSE EDUCATION: A CASE STUDY OF NATIONAL CYBER LEAGUE (NCL) PARTICIPATION}},
  booktitle = {Issues},
  year = {2022},
  doi = {10.48009/3\_iis\_2022\_111},
}
Figure