ChatGPT Participates in a Computer Science Exam

Sebastian Bordt, Ulrike von LuxburgarXiv 2023 · OpenReview

TL;DR

This paper investigates ChatGPT's performance on a real undergraduate computer science exam, finding that it narrowly passed. The results suggest that while ChatGPT can perform well on exam-like tasks, its success might be attributed to its training data rather than genuine understanding, highlighting the need for careful evaluation of AI capabilities.

What Problem Does This Paper Address?

As large language models (LLMs) like ChatGPT become increasingly sophisticated, a critical question arises: can they perform complex, real-world tasks that require specialized knowledge, such as passing university exams? While anecdotal evidence suggests impressive capabilities, rigorous, systematic evaluations are lacking, especially for exams that are not specifically curated for AI testing. This paper aims to address this gap by evaluating ChatGPT's performance on a standard undergraduate computer science exam in "Algorithms and Data Structures," a field requiring logical reasoning and problem-solving. The authors want to understand if ChatGPT can truly "understand" and apply CS concepts or if its performance is merely a sophisticated mimicry based on its vast training data.

Methods

The researchers designed an experiment where ChatGPT was tasked with answering all questions from a real undergraduate computer science exam. The exam covered topics like sorting algorithms, graph traversal, and dynamic programming, and included various question formats: multiple-choice, proofs, pseudo-code writing, and graph drawing.

To ensure a fair and unbiased evaluation, ChatGPT's answers were:

  1. Hand-copied: The digital responses were transcribed onto physical exam sheets, mimicking student submissions.
  2. Blindly Graded: These sheets were then mixed with those of 200 actual students and graded by teaching assistants who were unaware of which submission belonged to ChatGPT.
  3. Standardized Prompting: ChatGPT was prompted with clear instructions to provide short, precise answers, and the exam questions were presented in a simple, standardized format (including LaTeX source code for mathematical and graphical elements).

The study also included an evaluation of GPT-4's performance on the same exam, though this grading was not blind.

Datasets

The primary "dataset" for this study was a real undergraduate computer science exam in "Algorithms and Data Structures." The exam was designed for Bachelor of Science students in their second year and covered a standard curriculum. The authors emphasize that the exam questions are structurally similar to those found in other exams, homework problems, and online teaching materials, which are likely part of LLM training data. No new benchmark datasets were created for this study; instead, an existing curriculum assessment was used.

Key Insights and Ideas

  • Passing, but Barely: ChatGPT's ability to pass a university exam is impressive, but its narrow margin of success (20.5/40 points) suggests it's not a guaranteed ace.
  • Training Data is Key: The authors strongly caution against concluding that ChatGPT "understands" computer science. They highlight that the exam's content is heavily documented online, making it plausible that ChatGPT's success stems from memorizing and regurgitating information from its training data rather than genuine comprehension.
  • Structured Output Matters: ChatGPT excelled at tasks requiring structured output like pseudo-code but struggled with tasks involving visual representations like graphs, often producing incorrect or nonsensical diagrams.
  • GPT-4 Shows Improvement: The upgrade to GPT-4 demonstrated a significant performance boost, bringing its score closer to the average student performance, indicating ongoing progress in LLM capabilities.

Key Results

  • ChatGPT-3.5: Achieved a score of 20.5 out of 40 points, narrowly passing the exam (minimum passing score was 20). This placed it below the average student score of 23.9 points.
  • ChatGPT-4: Achieved an estimated score of 24 out of 40 points, representing a 17% improvement over GPT-3.5. This score is on par with the average student performance.
  • Performance Variation: ChatGPT performed well on exercises involving pseudo-code and simple proofs but struggled with questions requiring graph drawing or explaining the workings of standard algorithms.

Limitations and Future Work

The authors acknowledge several limitations:

  • Training Data Bias: The primary limitation is the potential for ChatGPT's performance to be heavily influenced by its training data, which likely includes similar exam content. This makes it difficult to ascertain true understanding.
  • Exam Specificity: While the exam is standard, it's still a single data point. Future work could involve testing ChatGPT on a wider range of exams and subjects.
  • Prompt Engineering: While the authors aimed for simple prompting, they acknowledge that prompt engineering can significantly impact LLM output.

The authors suggest that much more research is needed to understand the true abilities and limitations of LLMs. They emphasize the need for novel benchmarks to assess these models more comprehensively and to move beyond simply testing their ability to pass existing assessments. Future work could explore how LLMs can be used as tools to assist learning rather than just as potential replacements for human understanding.

Summary updated

Abstract

We asked ChatGPT to participate in an undergraduate computer science exam on ''Algorithms and Data Structures''. The program was evaluated on the entire exam as posed to the students. We hand-copied its answers onto an exam sheet, which was subsequently graded in a blind setup alongside those of 200 participating students. We find that ChatGPT narrowly passed the exam, obtaining 20.5 out of 40 points. This impressive performance indicates that ChatGPT can indeed succeed in challenging tasks like university exams. At the same time, the questions in our exam are structurally similar to those of other exams, solved homework problems, and teaching materials that can be found online and might have been part of ChatGPT's training data. Therefore, it would be inadequate to conclude from this experiment that ChatGPT has any understanding of computer science. We also assess the improvements brought by GPT-4. We find that GPT-4 would have obtained about 17\% more exam points than GPT-3.5, reaching the performance of the average student. The transcripts of our conversations with ChatGPT are available at \url{https://github.com/tml-tuebingen/chatgpt-algorithm-exam}, and the entire graded exam is in the appendix of this paper.
BibTeX
@misc{bordt2023chatgpt,
  author = {Sebastian Bordt and Ulrike von Luxburg},
  title = {{ChatGPT Participates in a Computer Science Exam}},
  year = {2023},
  eprint = {2303.09461},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.09461},
}
Figure