TL;DR
This paper provides a much-needed systematic review of how Artificial Intelligence (AI) and Machine Learning (ML) are being used in physics education research (PER). By analyzing 79 articles from key PER publications, the authors identify six major themes, highlighting the growing trend of using AI/ML for assessment development and analysis, as well as for predicting student success and engagement.
What Problem Does This Paper Address?
The rapid advancement of AI and ML technologies has led to their increasing adoption across various research fields, including physics education. However, despite this growing interest, there's been a lack of systematic documentation and overview of how these powerful tools are being applied within PER. This paper aims to fill that gap by mapping the existing literature, identifying the key areas where AI/ML techniques are being utilized, and providing a structured understanding of this evolving research landscape.
Methods
The authors conducted a systematic literature review following the PRISMA guidelines. They focused on two primary publication venues: the Physical Review Physics Education Research (PRPER) journal and the Physics Education Research Conference (PERC) proceedings. Their search strategy involved keywords such as "artificial intelligence," "machine learning," "AI," and "ML," as well as related terms like "Natural Language Processing" and "clustering." A total of 79 articles were included in the review. The classification of these articles into themes was primarily based on their abstracts, with full text consulted when abstracts were insufficient. The authors also performed a cross-check on a subset of articles to ensure the reliability of their classification.
Datasets
The paper does not explicitly list or introduce specific datasets used for training or evaluation within the reviewed articles. Instead, it focuses on the application of AI/ML techniques to existing physics education research data, which would vary greatly across the individual studies.
Key Insights and Ideas
- AI/ML is a rapidly growing area in PER: The number of articles employing AI/ML in PER has significantly increased in recent years, particularly in the PRPER journal.
- Assessment is a major focus: The most common application of AI/ML in PER is in the development, evaluation, and analysis of physics assessments, including student responses to various question types.
- Predicting student outcomes is key: Beyond assessments, AI/ML is being used to predict student success, analyze engagement patterns, and understand factors influencing admissions.
- Methodological guidance is emerging: A notable theme involves studies that provide theoretical and practical guidelines for using AI/ML in PER, helping researchers adopt these methods effectively.
- The field is still maturing: While AI/ML is being applied, the review highlights the need for more systematic documentation and a broader scope of publications to fully capture the landscape.
Key Results
The review identified six major themes in the application of AI/ML in PER:
- Development and analysis of assessments: This was the most prevalent theme, encompassing the use of AI/ML for characterizing student responses, developing and evaluating assessments with generative AI, and analyzing standardized assessments.
- Student success: Studies in this theme focused on developing predictive models to facilitate student success in various physics course formats (hybrid, flipped, remote) and identifying predictors of success.
- Student engagement: This theme covers research on student collaboration, interaction in online and hybrid environments, and the impact of learning environments on student performance and emotions.
- Theoretical and methodological overview: These studies offer guidance on using AI/ML for qualitative research, evaluating statistical models, and performing analyses like social network analysis.
- Admissions: A smaller theme, this includes research on using AI/ML to predict factors influencing university admissions and graduate program success.
- Others: This category encompasses literature synthesis, mapping pedagogical practices, and evaluating course transformations.
Limitations and Future Work
The authors acknowledge several limitations:
- Limited publication scope: The review was restricted to PRPER and PERC, potentially excluding relevant work from other journals and conferences.
- Geographic bias: The focus on US-based publications might miss international contributions.
- Abstract-based classification: While efforts were made to ensure accuracy, classifying primarily based on abstracts could lead to missed nuances.
- Definition of AI/ML: The study used a specific working definition of AI/ML, which might have excluded some relevant papers.
Future work suggested includes:
- Expanding the literature corpus: Including a wider range of journals and international sources.
- Refining the AI/ML definition: To ensure a more comprehensive inclusion of relevant studies.
- Deeper technical analysis: Focusing more on the specific algorithms and methods used in the papers.
- Quantitative analysis: Employing NLP techniques for large-scale topic analysis.
- Ethical considerations: Investigating the ethical implications of using AI/ML in education, especially with the rise of generative AI.
Summary updated