On the relevance of pre-neural approaches in natural language processing pedagogy

Aditya Joshi, Jake Renzella, Pushpak Bhattacharyya, Saurav Jha, Xiangyu Zhang

Abstract

While deep learning approaches represent the state-of-the-art of natural language processing (NLP) today, classical algorithms and approaches still find a place in NLP textbooks and courses of recent years. This paper discusses the perspectives of conveners of two introductory NLP courses taught in Australia and India, and examines how classical and deep learning approaches can be balanced within the lecture plan and assessments of the courses. We also draw parallels with the objects-first and objects-later debate in CS1 education. We observe that teaching classical approaches adds value to student learning by building an intuitive understanding of NLP problems, potential solutions, and even deep learning models themselves. Despite classical approaches not being state-of-the-art, the paper makes a case for their inclusion in NLP courses today.

Introduction: The Relevance Crisis

The rapid ascent of Large Language Models (LLMs) like ChatGPT has created a "relevance crisis" in Natural Language Processing (NLP) education. Students entering introductory courses are often eager to jump straight into building complex neural architectures, leading to skepticism regarding "obsolete" concepts such as Hidden Markov Models (HMMs), regular expressions, or N-grams. This paper, authored by researchers from the University of New South Wales (UNSW) and the Indian Institute of Technology Bombay (IITB), addresses the challenge of curriculum design in this era: how to integrate cutting-edge neural approaches without sacrificing the foundational principles that make those approaches understandable.

Methodology and Comparative Case Study

The researchers employed a comparative case study approach, analyzing two distinct introductory NLP courses:

  1. NLP-UNSW (Australia): A new course established in 2024 with 60 students.
  2. NLP-IITB (India): A long-standing course with 19 years of history and 150 students.

The study analyzed these curricula through the lens of Cognitive Load Theory, suggesting that simpler, rule-based models provide a "low-stakes" environment for students to grasp linguistic challenges before tackling the high complexity of neural architectures. Furthermore, the authors reviewed seven major NLP textbooks and syllabi from global institutions (e.g., CMU, NYU, Edinburgh) to benchmark the balance between classical and neural content.

A Pedagogical Framework for Intuition

The authors argue that classical methods are essential for teaching the "three pillars" of NLP: Linguistics (the "what" and "why"), Probability (the mathematical engine), and Implementation (the code). They propose a structured pedagogical flow that moves from explicit rules to probabilistic models, and finally to neural enhancements.

Figure 4: The proposed pedagogical flow from linguistic intuition to recent advancements.

This progression allows students to see the "path" a model takes. By starting with transparent, rule-based systems, the "black box" of a Transformer becomes less intimidating. The paper highlights two strategies for this balance: interleaving (teaching classical and neural methods for a specific task side-by-side) and sequential (moving through the spectrum over the semester).

Results and Student Outcomes

The findings suggest a strong student preference for foundational learning. A survey at IITB revealed that 80% of students preferred a course that builds intuition over one that simply provides a "suite of methods," and 67% valued learning fundamental approaches alongside state-of-the-art techniques.

Analysis of student projects further validated this approach. Even when given the freedom to use any tool, many teams incorporated classical techniques—such as Support Vector Machines (SVMs) or Naive Bayes—either for feature engineering or as interpretable baselines to measure the "performance jump" provided by neural models.

Figure 6: Word clouds showing the diversity of techniques used in student projects, spanning both deep-learning and classical domains.

Conclusion

Despite the "neural revolution," the consensus among modern NLP textbooks and the success of the UNSW and IITB courses suggest that classical foundations remain vital. By treating classical approaches as baselines and intuition-builders, educators can better prepare students to not only use modern AI but to understand and debug it. The paper concludes that while the "classical" portion of the curriculum may compress over time, its role in managing cognitive load and building linguistic intuition remains indispensable.

Summary updated

Abstract

While deep learning approaches represent the state-of-the-art of natural language processing (NLP) today, classical algorithms and approaches still find a place in NLP textbooks and courses of recent years. This paper discusses the perspectives of conveners of two introductory NLP courses taught in Australia and India, and examines how classical and deep learning approaches can be balanced within the lecture plan and assessments of the courses. We also draw parallels with the objects-first and objects-later debate in CS1 education. We observe that teaching classical approaches adds value to student learning by building an intuitive understanding of NLP problems, potential solutions, and even deep learning models themselves. Despite classical approaches not being state-of-the-art, the paper makes a case for their inclusion in NLP courses today.
BibTeX
@inproceedings{joshi2024relevance,
  author = {Aditya Joshi and Jake Renzella and Pushpak Bhattacharyya and Saurav Jha and Xiangyu Zhang},
  title = {{On the relevance of pre-neural approaches in natural language processing pedagogy}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}

Loading related content...

Figure