A Chameleon Effect of Inquiry-Based Science Teaching on Science Achievement: Evidence From PISA 2015

Xian WuProceedings of the 2019 AERA Annual Meeting 2019 · DOI

Abstract

Inquiry-based science teaching (IBST) is a cornerstone of modern science education, yet international assessments like PISA often show a negative correlation between its frequency and student performance. This study analyzes PISA 2015 data from the USA and the top 10 performing regions—including Singapore, Japan, Estonia, and China—to resolve this "Inquiry Paradox." By distinguishing between the levels of inquiry, the research uncovers a "Chameleon Effect": while guided, low-level inquiry promotes achievement, open, high-level inquiry is associated with lower scores.

Introduction: The Inquiry Paradox

For decades, science education has moved away from rote memorization toward inquiry-based teaching, where students engage in the practices of scientists. However, large-scale data from the Programme for International Student Assessment (PISA) has consistently reported a troubling negative relationship between the frequency of inquiry activities and science test scores. This paper addresses this paradox by questioning whether IBST should be treated as a monolithic concept or a spectrum of activities with varying impacts.

Methodology and Datasets

The researcher employed a two-step statistical approach to dissect the PISA 2015 data. First, Rasch Rating Scale Modeling was used to evaluate the 8-item IBST scale, allowing for a ranking of classroom activities by their "difficulty to endorse"—essentially, how common or rare they are in practice. Second, Multiple Linear Regression (MLR) estimated the effect of these activities on science achievement while controlling for socio-economic status (SES), gender, and student self-efficacy.

The study primarily utilized the PISA 2015 dataset, focusing on a USA sample of 5,128 students and a cross-cultural comparison of 70,532 students from the top 10 performing regions (including Singapore, Japan, Estonia, Canada, and parts of China). Historical context was provided by comparing these results against the PISA 2006 American dataset (5,480 students).

The Hierarchy of Inquiry

The Rasch analysis revealed a consistent "difficulty-to-endorse" order across all regions and time periods. Activities like students being given the chance to explain their ideas were the most common ("easiest" to endorse), while students designing their own experiments were the rarest ("hardest" to endorse).

Item map comparison between PISA 2015 and PISA 2006 showing the stable hierarchy of inquiry activities.

This hierarchy suggests a universal gradient of how inquiry is implemented in schools, regardless of cultural context or the decade of assessment.

The Chameleon Effect: Low-Level vs. High-Level Inquiry

The core finding of the paper is the "Chameleon Effect": IBST changes its impact based on its level of implementation. The author categorized the eight PISA items into two groups:

  • Low-Level IBST (Guided Inquiry): Includes activities like explaining ideas, applying concepts to life, and drawing conclusions.
  • High-Level IBST (Open Inquiry): Includes student-led activities like designing experiments, conducting investigations, and class debates.
Descriptions and coding of the eight inquiry-based science teaching items used in the PISA assessments.

The MLR results were striking. Low-level, teacher-supported inquiry had a significant positive effect on science scores ($\beta = 17.967$). Conversely, high-level, student-led inquiry had a significant negative effect ($\beta = -38.334$). This suggests that while guiding students through scientific reasoning is beneficial, asking them to lead complex experimental design without sufficient structure may be detrimental to test performance.

Conclusion and Implications

The study concludes that the "Inquiry Paradox" is a result of conflating different levels of inquiry. The negative correlation found in previous PISA reports is likely driven by the inclusion of high-level activities that are difficult to implement effectively.

A key limitation is that PISA measures the frequency of these activities, not their quality. High-level inquiry may fail not because the pedagogy is flawed, but because it requires a level of teacher expertise and classroom resources that are often lacking. Future research should focus on how to better prepare teachers for the complexities of open inquiry, ensuring that "science-like" activities translate into genuine scientific understanding.

Summary updated

Abstract

and regions ranking on the top 10 Singapore (5,694) Macao (3,935) Japan (6,416) Canada (17,668) Estonia (5,341) Viet Nam (5,822) Chinese Taipei (7,068) Hong Kong (3,794) Finland (5,620) China* (9,174) * China is represented by the provinces of Beijing, Shanghai, Jiangsu, and Guangdong.
BibTeX
@inproceedings{wu2019chameleon,
  author = {Xian Wu},
  title = {{A Chameleon Effect of Inquiry-Based Science Teaching on Science Achievement: Evidence From PISA 2015}},
  booktitle = {Proceedings of the 2019 AERA Annual Meeting},
  year = {2019},
  doi = {10.3102/1442545},
}
Figure