Abstract
While deep learning approaches represent the state-of-the-art of natural language processing (NLP) today, classical algorithms and approaches still find a place in NLP textbooks and courses of recent years. This paper discusses the perspectives of conveners of two introductory NLP courses taught in Australia and India, and examines how classical and deep learning approaches can be balanced within the lecture plan and assessments of the courses. We also draw parallels with the objects-first and objects-later debate in CS1 education. We observe that teaching classical approaches adds value to student learning by building an intuitive understanding of NLP problems, potential solutions, and even deep learning models themselves. Despite classical approaches not being state-of-the-art, the paper makes a case for their inclusion in NLP courses today.
Introduction: The Transformer-First Dilemma
In the era of ChatGPT and BERT, NLP educators face a significant dilemma: should they dedicate limited semester hours to "obsolete" rule-based and statistical methods, or jump straight into the state-of-the-art (SOTA)? Students often arrive with "Transformer-first" expectations, viewing classical methods as historical footnotes. However, jumping straight into complex neural architectures can lead to a "black box" understanding where students can use models but lack a grasp of the underlying linguistic or probabilistic challenges. This paper investigates how to strike a balance that satisfies student demand for modern skills while providing a rigorous theoretical foundation.
Comparative Methodology
The paper presents a comparative pedagogical study of two distinct introductory NLP courses:
- NLP-UNSW (University of New South Wales, Australia): A new course (first offered in 2024) that utilizes an "interleaved" method, introducing modern libraries early to build engagement before diving into foundational mechanics.
- NLP-IITB (Indian Institute of Technology Bombay, India): A long-standing course with 19 years of history, emphasizing a deep dive into fundamentals.
The authors analyzed curricula, assessment strategies, and student feedback, while also drawing on Cognitive Load Theory and parallels to the "objects-first vs. objects-later" debate in computer science education (CS1). They also surveyed seven major textbooks and six global university courses to understand current sequencing trends.
Key Insights: The Path to Intuition
A core finding is that teaching classical approaches adds value by building an intuitive understanding of NLP problems. Rule-based systems, for instance, demonstrate why NLP is difficult; when students fail to capture nuance with manual rules, they gain a visceral appreciation for statistical and neural models.
The authors argue that probability serves as a "common thread." Concepts like the Softmax function in a Transformer are more accessible if students already understand the probabilistic foundations of Hidden Markov Models (HMMs). This progression helps avoid cognitive overload by grounding complex abstractions in simpler, more transparent models.
Results: Valuing the Fundamentals
The study found strong student support for this balanced approach. At IITB, 80% of students preferred a course that builds intuition over one that simply provides a suite of modern methods, and 67% specifically valued learning fundamental approaches over just SOTA techniques.
In the UNSW course, requiring students to implement a classical baseline before moving to neural models led to more diverse and interpretable project submissions. Students successfully integrated a mix of techniques—such as SVMs and Naive Bayes alongside BERT and LoRA—resulting in more resource-efficient analyses. This hybrid approach ensures that students are not just consumers of SOTA models, but informed practitioners who understand the trade-offs between different methodologies.
Conclusion
While deep learning represents the current state-of-the-art, classical methods remain vital for building the linguistic and probabilistic intuition necessary for true mastery of the field. By balancing these approaches, educators can move students beyond "black-box" usage toward a deeper understanding of how and why modern NLP systems work. The study suggests that a foundation in classical NLP is not a detour, but a necessary step in developing robust expertise in the age of Large Language Models.
Summary updated