Degree Planning with PLAN-BERT: Multi-Semester Recommendation Using Future Courses of Interest

Erzhuo Shao, Shiyuan Guo, Zachary A. PardosAAAI 2021 · DOI

Abstract

PLAN-BERT is a transformer-based recommendation system designed for long-term degree planning. By leveraging bidirectional context and future "anchor" courses, it addresses error accumulation in multi-step prediction and the user-wise cold start problem. Experimental results on large-scale university datasets demonstrate significant improvements in recommendation accuracy and personalization over traditional sequential models.

Introduction

Most recommender systems focus on "next-item" or "next-basket" prediction. However, high-stakes scenarios like academic degree planning require consecutive multi-step prediction. Standard auto-regressive models, such as LSTMs, predict one step at a time, using their own previous outputs as inputs for subsequent steps. This often leads to "error accumulation," where early mistakes propagate through a four-year plan. Furthermore, these models struggle with "cold starts"—recommending courses for freshmen with no academic history. PLAN-BERT addresses these issues by utilizing "anchors"—specific future courses a student intends to take—to stabilize and personalize long-term recommendations.

Methodology

The authors adapted the BERT (Bidirectional Encoder Representations from Transformers) architecture for degree planning. BERT’s masked language modeling objective is well-suited for this task, where some semesters are "known" (past history or future interests) and others are "missing" (to be recommended).

The architecture and input/prediction targets of PLAN-BERT.

Key Innovations: * Non-Auto-regressive Prediction: PLAN-BERT predicts the entire remaining curriculum simultaneously, avoiding the error propagation inherent in sequential models. * Future Reference Items: The model accepts "future courses of interest" as input. These act as anchors that the self-attention mechanism uses to "fill in" the gaps. * Multi-Modal Inputs: The model incorporates relative semesters (positional encoding), semester seasons (Fall, Spring, Summer), and user/item features (e.g., major, department). * Training: A two-stage process involving pre-training on random masks and fine-tuning on realistic "past + future" scenarios.

Datasets

The study utilized two massive, anonymized datasets:

  1. UNIVERSITY1: 4.6 million course enrollments from 134,275 students at UC Berkeley (2008–2017).
  2. SYSTEM1: 10.4 million enrollments from over 900,000 students across 19 colleges in the City University of New York (CUNY) system.

Key Insights and Ideas

  • The Power of "Anchors": Providing just two future courses of interest improved recommendation accuracy for freshmen by 120%. This suggests that user intent is a stronger signal than historical averages for new users.
  • BERT vs. BiLSTM: BERT’s self-attention mechanism is superior at weighing the importance of specific future "anchor" courses against past prerequisites compared to BiLSTMs.
  • Personalization: In user studies, students favored PLAN-BERT for its "specificity"—how tailored the plan felt to their individual goals—over mere logical "sensibility."

Key Results

PLAN-BERT outperformed all baselines, including UserKNN, LSTM, and BiLSTM. On the UNIVERSITY1 dataset, PLAN-BERT achieved a Recall@10 of 0.2859 for freshmen, compared to 0.1299 for a standard LSTM.

Performance comparison on UNIVERSITY1 dataset showing Recall@10 vs. number of reference courses.

Adding user and item features provided a substantial boost, particularly for upperclassmen with specialized course requirements. In an online study with 63 students, PLAN-BERT achieved a 72.13% specificity rating, significantly higher than the UserKNN baseline (65.57%) and approaching the specificity of actual graduated student paths (77.05%).

Limitations and Future Work

While PLAN-BERT shows promise, it currently excludes course grades, which can influence enrollment eligibility. Future work could explore grade prediction and address potential distribution shifts in student interests. Beyond academia, the PLAN-BERT framework is applicable to other planning domains, such as travel itinerary planning or long-term e-commerce project recommendations.

Summary updated

BibTeX
@inproceedings{shao2021degree,
  author = {Erzhuo Shao and Shiyuan Guo and Zachary A. Pardos},
  title = {{Degree Planning with PLAN-BERT: Multi-Semester Recommendation Using Future Courses of Interest}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i17.17751},
}

Loading related content...

Figure