Contemporary NLP Modeling in Six Comprehensive Programming Assignments

Greg Durrett, Jifan Chen, Shrey Desai, Tanya Goyal, Lucas Kabela, Yasumasa Onoe, Jiacheng Xu2021 · DOI

Abstract

We present a series of six programming assignments designed to teach the design and implementation of modern NLP systems to students ranging from advanced undergraduates to PhD candidates. These assignments emphasize a "full-stack" understanding, moving from hand-coded gradients in linear models to sophisticated neural architectures in PyTorch. The curriculum covers a broad spectrum of techniques, from industry-standard linear models to research-oriented deep learning, while remaining computationally tractable and fully autogradable.

Introduction

Teaching modern Natural Language Processing (NLP) requires balancing high-level library proficiency with a deep understanding of underlying algorithmic mechanics. "Neural-first" approaches often rely on high-level abstractions that obscure optimization details, while traditional linguistics-focused courses may neglect the engineering challenges of state-of-the-art systems. This paper introduces a curriculum that bridges this gap through a sequence of six assignments that build complexity incrementally.

The Six-Assignment Curriculum

The core of the proposed curriculum is a sequence of tasks (A1–A6) that transition students from manual implementation to the use of modern deep learning frameworks.

Overview of the six programming assignments, their core concepts, and architectural components.
  1. A1: Linear Sentiment Analysis: Students implement Stochastic Gradient Descent (SGD) and bag-of-words models from scratch, ensuring they understand the fundamentals of optimization before moving to libraries.
  2. A2: Neural Sentiment Analysis: The transition to PyTorch begins here, with students building Deep Averaging Networks (DANs) and integrating pre-trained GloVe embeddings.
  3. A3: Structured Prediction (NER): This assignment focuses on Hidden Markov Models (HMMs) and Conditional Random Fields (CRFs) for Named Entity Recognition, emphasizing linguistic structure and ambiguity.
  4. A4: Language Modeling: Students implement character-level Recurrent Neural Networks (RNNs), focusing on neural sequence modeling.
  5. A5: Semantic Parsing: This task involves sequence-to-sequence (seq2seq) models with attention to translate natural language into executable logical forms (e.g., GeoQuery).
  6. A6: Reading Comprehension: The final assignment tasks students with building a simplified version of models like BiDAF to extract answer spans from the SQuAD dataset.

To keep the focus on algorithmic logic, the code is intentionally "underengineered," avoiding complex Pythonic abstractions that might confuse students.

Curriculum Integration

To support the programming tasks, the curriculum is mapped to standard NLP textbooks, providing students with the necessary theoretical background. This ensures that the practical implementation is grounded in established literature.

Mapping of assignments to chapters in Eisenstein and Jurafsky + Martin textbooks.

Pedagogical Insights and Tractability

A primary challenge in modern NLP education is the computational cost of training large models. The authors achieve tractability by selecting specific tasks—such as character-level language modeling and small-scale semantic parsing—that keep vocabulary sizes small and training times short. This allows students to iterate quickly without requiring massive GPU clusters.

Furthermore, the curriculum addresses the "prior gap": the tendency for students to lack an intuition for expected system behavior. By emphasizing debugging strategies and providing clear performance benchmarks, the assignments help students distinguish between difficult tasks and implementation bugs.

Results and Scalability

The curriculum has been tested on over 300 students across multiple academic levels. Its modular design allows instructors to customize the difficulty; for instance, undergraduate courses might focus on the earlier assignments, while PhD courses engage with the more complex structured prediction and parsing tasks. Additionally, assignments A1 through A5 are fully autogradable via platforms like Gradescope, making them viable for large-scale enrollment.

Limitations

While comprehensive, the curriculum currently lacks explicit implementation of transition-based or chart-based parsing due to engineering complexity. Furthermore, while Transformers are discussed in lectures, they are not yet part of the core implementation requirements to keep the coding burden manageable. Future iterations may also incorporate formal assessments of ethics and bias, which are currently handled through class discussion.

Summary updated

Abstract

We present a series of programming assignments, adaptable to a range of experience levels from advanced undergraduate to PhD, to teach students design and implementation of modern NLP systems. These assignments build from the ground up and emphasize fullstack understanding of machine learning models: initially, students implement inference and gradient computation by hand, then use Py-Torch to build nearly state-of-the-art neural networks using current best practices. Topics are chosen to cover a wide range of modeling and inference techniques that one might encounter, ranging from linear models suitable for industry applications to state-of-theart deep learning models used in NLP research. The assignments are customizable, with constrained options to guide less experienced students or open-ended options giving advanced students freedom to explore. All of them can be deployed in a fully autogradable fashion, and have collectively been tested on over 300 students across several semesters.
BibTeX
@misc{durrett2021contemporary,
  author = {Greg Durrett and Jifan Chen and Shrey Desai and Tanya Goyal and Lucas Kabela and Yasumasa Onoe and Jiacheng Xu},
  title = {{Contemporary NLP Modeling in Six Comprehensive Programming Assignments}},
  year = {2021},
  doi = {10.18653/v1/2021.teachingnlp-1.17},
}

Loading related content...

Figure