TL;DR
This paper investigates ChatGPT's performance on a real undergraduate computer science exam, finding that it narrowly passed. The results suggest that while ChatGPT can perform well on exam-like tasks, its success might be attributed to its training data rather than genuine understanding, highlighting the need for careful evaluation of AI capabilities.
What Problem Does This Paper Address?
As large language models (LLMs) like ChatGPT become increasingly sophisticated, a critical question arises: can they perform complex, real-world tasks that require specialized knowledge, such as passing university exams? While anecdotal evidence suggests impressive capabilities, rigorous, systematic evaluations are lacking, especially for exams that are not specifically curated for AI testing. This paper aims to address this gap by evaluating ChatGPT's performance on a standard undergraduate computer science exam in "Algorithms and Data Structures," a field requiring logical reasoning and problem-solving. The authors want to understand if ChatGPT can truly "understand" and apply CS concepts or if its performance is merely a sophisticated mimicry based on its vast training data.
Methods
The researchers designed an experiment where ChatGPT was tasked with answering all questions from a real undergraduate computer science exam. The exam covered topics like sorting algorithms, graph traversal, and dynamic programming, and included various question formats: multiple-choice, proofs, pseudo-code writing, and graph drawing.
To ensure a fair and unbiased evaluation, ChatGPT's answers were:
- Hand-copied: The digital responses were transcribed onto physical exam sheets, mimicking student submissions.
- Blindly Graded: These sheets were then mixed with those of 200 actual students and graded by teaching assistants who were unaware of which submission belonged to ChatGPT.
- Standardized Prompting: ChatGPT was prompted with clear instructions to provide short, precise answers, and the exam questions were presented in a simple, standardized format (including LaTeX source code for mathematical and graphical elements).
The study also included an evaluation of GPT-4's performance on the same exam, though this grading was not blind.
Datasets
The primary "dataset" for this study was a real undergraduate computer science exam in "Algorithms and Data Structures." The exam was designed for Bachelor of Science students in their second year and covered a standard curriculum. The authors emphasize that the exam questions are structurally similar to those found in other exams, homework problems, and online teaching materials, which are likely part of LLM training data. No new benchmark datasets were created for this study; instead, an existing curriculum assessment was used.
Key Insights and Ideas
- Passing, but Barely: ChatGPT's ability to pass a university exam is impressive, but its narrow margin of success (20.5/40 points) suggests it's not a guaranteed ace.
- Training Data is Key: The authors strongly caution against concluding that ChatGPT "understands" computer science. They highlight that the exam's content is heavily documented online, making it plausible that ChatGPT's success stems from memorizing and regurgitating information from its training data rather than genuine comprehension.
- Structured Output Matters: ChatGPT excelled at tasks requiring structured output like pseudo-code but struggled with tasks involving visual representations like graphs, often producing incorrect or nonsensical diagrams.
- GPT-4 Shows Improvement: The upgrade to GPT-4 demonstrated a significant performance boost, bringing its score closer to the average student performance, indicating ongoing progress in LLM capabilities.
Key Results
- ChatGPT-3.5: Achieved a score of 20.5 out of 40 points, narrowly passing the exam (minimum passing score was 20). This placed it below the average student score of 23.9 points.
- ChatGPT-4: Achieved an estimated score of 24 out of 40 points, representing a 17% improvement over GPT-3.5. This score is on par with the average student performance.
- Performance Variation: ChatGPT performed well on exercises involving pseudo-code and simple proofs but struggled with questions requiring graph drawing or explaining the workings of standard algorithms.
Limitations and Future Work
The authors acknowledge several limitations:
- Training Data Bias: The primary limitation is the potential for ChatGPT's performance to be heavily influenced by its training data, which likely includes similar exam content. This makes it difficult to ascertain true understanding.
- Exam Specificity: While the exam is standard, it's still a single data point. Future work could involve testing ChatGPT on a wider range of exams and subjects.
- Prompt Engineering: While the authors aimed for simple prompting, they acknowledge that prompt engineering can significantly impact LLM output.
The authors suggest that much more research is needed to understand the true abilities and limitations of LLMs. They emphasize the need for novel benchmarks to assess these models more comprehensively and to move beyond simply testing their ability to pass existing assessments. Future work could explore how LLMs can be used as tools to assist learning rather than just as potential replacements for human understanding.
Summary updated