Design of an Evaluation System for Disruptive Technologies to Benefit Smart Cities

Jun Qiu, Jing Cao, Xinyi Gu, Zimo Ge, Zhe Wang, Liang ZhengSustainability 2023 · DOI

TL;DR

This paper introduces a novel framework for identifying and evaluating disruptive technologies crucial for smart city development. By combining the Analytic Hierarchy Process (AHP) and Entropy Weight Method, it provides a structured approach to assess technologies' potential impact, offering valuable insights for urban planning and innovation.

What Problem Does This Paper Address?

The rapid evolution of cities towards "smart cities" is heavily driven by technological advancements. However, a significant challenge lies in identifying and evaluating "disruptive technologies" – innovations that fundamentally alter industries and create new markets. While the concept of disruptive innovation is well-established, there's a lack of systematic methods to pinpoint and assess these technologies specifically within the context of smart city development. This gap makes it difficult for cities to proactively plan, mitigate risks, and strategically invest in technologies that will truly benefit their future. This paper aims to fill this void by proposing a robust evaluation system.

Methods

The core of this research is a methodological framework designed to identify and evaluate disruptive technologies for smart cities. It involves three main steps:

  1. Feature Extraction: The researchers first analyzed historical case studies of disruptive technologies (e.g., the Internet, silicon transistors, AC transmission) to extract their common features and understand their impact.
  2. Criteria Proposal: Based on these features and the requirements of smart cities, a set of criteria for selecting disruptive technologies was proposed. These criteria focus on the technology's ability to bring significant improvement/transformation, cause revolutionary changes in production and life, and have a large potential for application or impact.
  3. Evaluation System Design: This is the most significant contribution. The paper develops an evaluation system using a combination of:
  4. Analytic Hierarchy Process (AHP): Used to establish a hierarchical structure of criteria and sub-criteria and determine their relative importance through pairwise comparisons by experts.
  5. Entropy Weight Method: Applied to objectively determine the weights of indicators based on the amount of information they contain, complementing the subjective expert judgments from AHP.

The evaluation system comprises 5 primary indicators and 11 secondary indicators, designed to be comprehensive and unbiased. The feasibility of this framework was then tested using the field of information science as a case study.

Datasets

The paper does not utilize traditional datasets for training or evaluation in the machine learning sense. Instead, it relies on:

  • Expert Judgment: Data was collected from 231 experts (professors, senior researchers) from universities, research institutions, and enterprises. These experts provided scores and participated in pairwise comparisons for the AHP.
  • Historical Case Studies: The initial feature extraction phase drew upon well-documented historical examples of disruptive technologies.

The paper does not introduce a new benchmark dataset.

Key Insights and Ideas

  • The "Why" Behind Disruption: The research emphasizes that disruptive technologies aren't just incremental improvements; they fundamentally change how things are done, often by making them simpler, cheaper, or more accessible, leading to revolutionary shifts.
  • Holistic Evaluation is Key: Simply looking at technological advancement isn't enough. The framework highlights the importance of considering factors like societal impact, economic value generation, and the potential for cross-field applications when evaluating disruptive technologies for smart cities.
  • Bridging Subjective and Objective: The combination of AHP (expert-driven, subjective) and Entropy Weight Method (data-driven, objective) offers a more robust and balanced approach to evaluating complex technological phenomena.
  • Information Science as a Proving Ground: The choice of information science as a case study is strategic, given its central role in smart city development. The successful application here suggests the framework's potential for broader use.

Key Results

The study successfully developed and verified a framework for evaluating disruptive technologies.

  • Indicator System: A system with 5 primary and 11 secondary indicators was established.
  • Expert Consensus: The AHP analysis showed that indicators related to "great changes in technology, industry, models, etc." (C1) and "interactive features of the information society" (C4) were deemed most important among primary indicators. For secondary indicators, those related to "intelligent system and terminal interaction," "sharing data or applications," and "future product potential" received higher weights.
  • Case Study Findings: In the information science field, technologies like "Photoelectric integrated brain chip technology," "Superhuman vision for multi-dimensional information perception," and "Ultra-fast full-time vision model and chip" were ranked highest based on the evaluation.
  • Framework Validation: The Shapiro-Wilk test confirmed that the distribution of technology scores conformed to a normal distribution, supporting the reliability of the model's output.

Limitations and Future Work

  • Expert Dependency: The AHP method, while powerful, relies heavily on expert judgment. The paper acknowledges that the consistency of these judgments (measured by C.R.) needs careful attention, as inconsistencies can require significant readjustment.
  • Scope of Case Study: While the framework was tested in information science, its applicability to other fields might require adaptation of specific indicators.
  • Future Research Directions: The authors suggest exploring evaluation mechanisms that combine expert evaluation with quantitative assessment throughout the entire technology lifecycle. They also propose conducting sensitivity analyses to identify the framework's strengths and weaknesses and to optimize it. Generalizing the approach to multiple applications and further enhancing its sustainability assessment capabilities are also highlighted as future avenues.

Summary updated

Abstract

Technological empowerment has facilitated the development of cities, which have progressed from pre-industrial to industrial to information-based and are currently transitioning towards the advanced stage of smart cities. The evolution and transformation of cities are fuelled by technology, which serves as a key driver. Disruptive technologies are radically scientific innovations that dramatically change the way consumers, businesses, and industries operate by destroying the value of existing technical competencies, thereby providing organisations with the capability or technical foundation to alter their business environments. To ensure that a city has a clear understanding of its smart city development direction, it is crucial to establish a scientifically valid and reliable evaluation index and method to analyse and recognise the disruptive technologies closely related to industrial development, transformation, and competitiveness in smart cities. However, there is a paucity of study on this topic. This paper addresses this research gap by developing a framework for disruptive technology identification and evaluation for smart cities using an entropy weight method and analytic hierarchy process. The evaluation index system contains 5 primary indicators and 11 secondary indicators according to the connotation of disruptive technologies in smart cities. The feasibility and effectiveness of the proposed framework are verified in the field of information science. This study provides technical knowledge and theoretical support for the evaluation and construction of smart cities.
BibTeX
@inproceedings{qiu2023design,
  author = {Jun Qiu and Jing Cao and Xinyi Gu and Zimo Ge and Zhe Wang and Liang Zheng},
  title = {{Design of an Evaluation System for Disruptive Technologies to Benefit Smart Cities}},
  booktitle = {Sustainability},
  year = {2023},
  doi = {10.3390/su15119109},
}
Figure