Research on the evaluation of science and technology museums

Xu Yang, Meili Wang, Qi Liu, Shujing Wang, Ying Zhang, Yang ZhaoModern 2025 · DOI

TL;DR

This paper proposes a robust, multi-dimensional evaluation framework for assessing the social impact and operational efficiency of science and technology (S&T) museums. By applying the TOPSIS method to data from 408 Chinese museums, the researchers demonstrate that while scale often dictates overall ranking, "small but refined" institutions can outperform larger peers in specific service metrics.

What Problem Does This Paper Address?

Science and technology museums are cornerstones of public scientific literacy, yet evaluating their success has traditionally been a "blunt instrument" exercise. Most assessments rely on static metrics like floor space or raw visitor counts, which fail to capture dynamic social benefits, research output, or public satisfaction.

Furthermore, the "digital divide" in science communication—where traditional physical exhibits meet new online platforms—is rarely accounted for in a standardized way. The authors set out to create a scientific, fair, and quantitative system that can compare diverse institutions—from massive national landmarks to specialized regional centers—on a level playing field.

Methods

The researchers developed a four-level indicator system weighted through expert consultation and the Analytic Hierarchy Process (AHP). To rank the museums, they employed the TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution) method.

  • Why TOPSIS? Unlike simple weighted averages, TOPSIS identifies the theoretical "ideal" (best possible scores across all metrics) and "anti-ideal" (worst possible scores) solutions. It then ranks each museum based on its geometric distance to these two points. This approach is particularly effective for handling a mix of positive indicators (e.g., visitor satisfaction) and negative indicators (e.g., complaint rates).
  • The Framework: The system is built on three primary pillars:
  • Social Education & Public Participation (50% weight): Focuses on exhibition impact, educational activities, and digital outreach.
  • Research Innovation & Social Contribution (30% weight): Measures the development of new exhibits, academic research, and industry influence.
  • Service Efficiency & Social Satisfaction (20% weight): Evaluates operational efficiency (visitors per square meter) and public feedback.

Datasets

The study utilizes a comprehensive dataset from 2021, covering 408 "qualified" S&T museums across China. To be included, museums had to meet specific national standards, such as having at least 1,000 square meters of permanent exhibition space.

The sample was categorized by scale:

  • Extra-large: 19 museums
  • Large: 41 museums
  • Medium: 81 museums
  • Small: 267 museums

Key Insights and Ideas

  • The "Scale Advantage" is Real but Not Absolute: Unsurprisingly, the top 10 museums nationally are all "Extra-large" or "Large." These institutions benefit from concentrated resources, talent, and regional prominence.
  • "Small but Refined" (小而精): A fascinating finding is that the "head" of the small museum category actually outscores many medium-sized museums. In regions like East China, small museums are successfully carving out niches by focusing on high-quality, specialized services rather than trying to compete on scale.
  • The Innovation Barrier: The "Research and Innovation" metric proved to be the biggest hurdle for smaller institutions. High R&D costs and long development cycles for new exhibits mean that smaller museums often rely on "off-the-shelf" content, limiting their social contribution scores.
  • Regional Clustering: High-performing museums are distributed across China, but the most successful "small" museums are heavily clustered in developed coastal regions, suggesting that local economic health and "talent density" are critical secondary factors for museum success.

Key Results

  • Top Performer: Museum A1 (an extra-large facility in North China) took the top spot with a score of 0.6091, excelling particularly in exhibit R&D and radiation of social influence.
  • The Long Tail: While the "head" of the small museum category is competitive, the "tail" is long. Among the bottom 100 museums in the study, 85% were small institutions, highlighting a significant gap in resource allocation or management efficiency at the local level.
  • Consistency in the Middle: Medium-sized museums showed the least variance in scores, suggesting a more standardized (and perhaps less innovative) operational model compared to the highly variable small museum sector.

Limitations and Future Work

The authors acknowledge several areas for improvement:

  • Static vs. Dynamic Data: The current model uses a snapshot (2021 data). Future iterations should incorporate real-time monitoring to capture seasonal trends and long-term growth.
  • Digital Integration: While the paper includes "network coverage," the authors note that the split between physical and digital science communication is blurring. Future models need more nuanced metrics for "online-to-offline" (O2O) engagement.
  • Subjectivity in Weighting: Although expert scoring was used, the weights assigned to indicators remain somewhat subjective. The authors suggest exploring "Fuzzy Comprehensive Evaluation" to better handle the inherent ambiguity in social impact metrics.

Summary updated

Abstract

With the increasing importance of science and technology museums in public education and scientific dissemination, accurately assessing the operational effectiveness and social impact of these museums has become a significant issue. This paper constructs a comprehensive evaluation system and conducts a quantitative assessment of qualified science and technology museums. The evaluation indicators cover multiple dimensions, including social feedback, organizational management, exhibition and educational content, and development potential. The research results reveal differences among the museums in these dimensions. The findings offer valuable references for improving the scientificity and fairness of science and technology museum evaluations and provide theoretical support for related decision-making.
BibTeX
@inproceedings{yang2025research,
  author = {Xu Yang and Meili Wang and Qi Liu and Shujing Wang and Ying Zhang and Yang Zhao},
  title = {{Research on the evaluation of science and technology museums}},
  booktitle = {Modern},
  year = {2025},
  doi = {10.3724/j.issn.1671-4342.20250035},
}
Figure