The Advantages and Disadvantages of ChatGPT in Humanities, Social Sciences and STEM Education: From a Comparative Perspective

Yuman Wang, Shiqi WangLecture Notes in Education Psychology and Public Media 2024 · DOI

This academic paper, published in the Proceedings of the International Conference on Global Politics and Socio-Humanities, explores the pedagogical impact of Large Language Models (LLMs). Specifically, it proposes a comparative qualitative framework to evaluate how students in different academic tracks—Humanities/Social Sciences versus STEM—navigate the benefits and pitfalls of using ChatGPT.

TL;DR

The paper outlines a qualitative research design to compare the advantages and disadvantages of ChatGPT across different academic disciplines. Grounded in cognitivism, the study investigates how students use AI to manage cognitive load and the metacognitive processes involved in integrating AI-generated content into their education.

What Problem Does This Paper Address?

While the rapid adoption of ChatGPT in education is well-documented, the authors argue that there is a "notable gap" in understanding how its impact varies by field of study. STEM students might use AI for data analysis and coding, while Humanities students might use it for translation or essay drafting.

The paper seeks to move beyond general observations of "AI in the classroom" to pinpoint the specific "similarities and differences" in user experience between these two major academic branches. It also addresses the ethical tension students face: the desire for academic efficiency versus the risk of plagiarism and the loss of "human contact" in learning.

Methods

The authors employ a qualitative research design rooted in cognitivism theory. This theoretical lens focuses on internal mental structures and "metacognition" (thinking about one's own thinking).

The methodological approach consists of two phases:

  1. Open-ended Surveys: A 10-question questionnaire designed to categorize student experiences into "before" and "after" using ChatGPT.
  2. Semi-structured Interviews: In-depth, 30-minute individual sessions with a subset of participants to explore personalized experiences and emotional responses to AI tools.
  3. Thematic Analysis: The researchers use an iterative process to identify, code, and refine themes from the interview transcripts to draw comparative conclusions.

Datasets

The study does not use a pre-existing computational dataset. Instead, it generates a primary dataset through non-representative, purposeful sampling.

  • Participants: 15–20 students.
  • Distribution: An even split between high school and undergraduate students, and an even split between those studying Humanities/Social Sciences and STEM (8 participants per category).
  • Geographic Scope: Students from two provinces in China.

Key Insights and Ideas

  • AI as a Cognitive Load Reducer: The paper frames the decision to use ChatGPT as a metacognitive strategy. Students weigh the benefits of AI assistance against independent problem-solving to manage their "cognitive load," allowing them to focus on higher-level tasks.
  • The Metacognitive "Paraphrase" Loop: A significant insight is the description of the student workflow: receiving AI output, experiencing "metacognitive apprehension" regarding plagiarism, and then intentionally paraphrasing or modifying the text to meet academic standards.
  • Disciplinary Divergence: Preliminary literature review within the paper suggests that while ChatGPT shows "exceptional outcomes" in economics and critical thinking, its performance in specialized fields like medical education, law, and high-level mathematics is often "not totally sufficient."
  • The Human Gap: A recurring theme is that generative models lack "full conceptual comprehension" and cannot replicate the nuanced interaction provided by human professors or tutors.

Key Results

As this paper focuses heavily on the research framework and preliminary qualitative design, the "results" are presented as the identified categories of impact:

  • Advantages identified: Automatic essay grading, personalized feedback, and the ability to analyze large STEM datasets to identify patterns that traditional methods might miss.
  • Disadvantages identified: Generation of imprecise/misleading information, lack of human-centric empathy, and ethical concerns regarding authorship and transparency.
  • Comparative Goal: The study establishes that while the convenience of AI is a universal advantage, the risks (such as the "AI index" in plagiarism detectors like Turnitin) manifest differently depending on whether the student is submitting a creative essay or a technical report.

Limitations and Future Work

The authors candidly acknowledge several limitations:

  • Self-Report Bias: Data relies on student memory and honesty, which can be influenced by social desirability bias.
  • Timeliness: The study specifically looks at ChatGPT-3.5. Given the rapid release cycle of LLMs (like GPT-4 and beyond), the findings may have a short shelf-life.
  • Generalizability: The small sample size (15-20 students) and specific geographic focus (China) mean the results may not apply to students in different cultural or educational systems.
  • Future Directions: The authors suggest expanding this research to include the fields of arts and sports and conducting deeper investigations into the long-term societal impacts, such as potential increases in unemployment due to AI-driven automation.

Summary updated

Abstract

ChatGPT is a powerful language model capable of understanding and generating human language. The increasing number of students utilizing ChatGPT to support their academic endeavors has gained prominence, prompting discussions surrounding technology and its impact on education. With the emerging trend of ChatGPT, students now could enjoy a great deal of conveniences, but there are also a number of problems while using ChatGPT. Despite extensive research investigating the correlation between ChatGPT and education, there remains a notable gap in comprehensively examining the similarities and differences concerning the benefits and drawbacks of humanities, social sciences compared to STEM fields. This research intends to delve into these domains, employing qualitative analysis to acquire pertinent data. This research will be rested on cognitivism theory and the methodological approach will be aligned to the theory. This approach involves a combination of an open-ended survey and semi-structured interviews for high school and undergraduate students to holistically explore these facets. The finding of this research might enlighten some students in using technological tools in their study in the future.
BibTeX
@inproceedings{wang2024advantages,
  author = {Yuman Wang and Shiqi Wang},
  title = {{The Advantages and Disadvantages of ChatGPT in Humanities, Social Sciences and STEM Education: From a Comparative Perspective}},
  booktitle = {Lecture Notes in Education Psychology and Public Media},
  year = {2024},
  doi = {10.54254/2753-7048/46/20230649},
}
Figure