Three IQs of AI Systems and their Testing Methods

Feng Liu, Yong Shi, Ying Liu

Abstract

The rapid development of artificial intelligence has brought the artificial intelligence threat theory as well as the problem about how to evaluate the intelligence level of intelligent products. Both need to find a quantitative method to evaluate the intelligence level of intelligence systems, including human intelligence. Based on the standard intelligence system and the extended Von Neumann architecture, this paper proposes General IQ, Service IQ and Value IQ evaluation methods for intelligence systems, depending on different evaluation purposes. Among them, the General IQ of intelligence systems is to answer the question of whether the artificial intelligence can surpass the human intelligence, which is reflected in putting the intelligence systems on an equal status and conducting the unified evaluation. The Service IQ and Value IQ of intelligence systems are used to answer the question of how the intelligent products can better serve the human, reflecting the intelligence and required cost of each intelligence system as a product in the process of serving human.

1. Introduction

As artificial intelligence (AI) systems like AlphaGo achieve milestones such as defeating human champions, the discourse has split between the "AI threat theory" and the rapid commercialization of intelligent products. A critical missing component in this evolution is a rigorous, quantitative methodology to measure "intelligence" across diverse systems. Traditional benchmarks, such as the Turing Test, are often criticized for their subjectivity and focus on human-mimicry rather than absolute cognitive capacity. This paper addresses this gap by proposing a mathematical and architectural basis for AI evaluation, defining three distinct IQ metrics: General IQ, Service IQ, and Value IQ.

2. Methodology and Architecture

The proposed framework is built upon two theoretical pillars: the Standard Intelligence Model and the Extended Von Neumann Architecture.

2.1 The Standard Intelligence Model

An intelligent agent is defined by four core capabilities:

  • Knowledge Acquisition: The ability to receive information from the environment.
  • Knowledge Mastery: The ability to store and process acquired information.
  • Knowledge Creation: The ability to generate new knowledge or discover laws (innovation).
  • Knowledge Feedback: The ability to apply knowledge to interact with the environment.

2.2 Extended Von Neumann Architecture

To better model modern AI, the authors update the classical Von Neumann architecture by adding an "Innovation Generator" and "Cloud Storage." The Innovation Generator represents the system's ability to create new knowledge, while Cloud Storage acknowledges the networked nature of modern intelligence, allowing for collective knowledge sharing across multiple systems.

The Extended Von Neumann Architecture, incorporating cloud storage and innovation generation components to model modern intelligent systems.

3. Evaluation Metrics: The Three IQs

The framework categorizes intelligence evaluation into three distinct metrics based on the purpose of the assessment:

  • General IQ (AI G IQ): This metric treats AI and humans as equal agents. It utilizes a unified "AI IQ Test Scale" to determine if AI can surpass human intelligence in raw cognitive tasks.
  • Service IQ (AI S IQ): This focuses on the utility of an AI product to humans. It incorporates constraints such as law, ethics, and safety, measuring how well a system performs as a service.
  • Value IQ (AI V IQ): An economic metric defined as the ratio of Service IQ to price ((Service IQ / Price) * 100). This helps consumers and organizations assess the cost-effectiveness of intelligent systems.

4. Results and Discussion

The researchers applied the AI IQ Test Scale to evaluate several prominent AI systems against human benchmarks at various developmental stages (ages 6, 12, and 18).

Absolute IQ scores from 2014-2016 comparing human developmental baselines with leading AI systems like Google and Baidu.

The results reveal a significant "Innovation Gap." While AI systems demonstrate high proficiency in knowledge acquisition and mastery (e.g., calculation and translation), they struggle with creation and discovery. In 2016, the highest-ranked AI system, Google, achieved an Absolute IQ of 47.28—notably lower than the 55.5 score of a 6-year-old human. However, longitudinal data from 2014 to 2016 shows rapid improvement, indicating that AI is steadily closing the gap with human cognition.

5. Conclusion and Future Work

This paper provides a standardized framework for the quantitative evaluation of intelligence. By distinguishing between raw cognitive ability (General IQ) and applied utility (Service IQ), the model offers a nuanced view of AI progress. Future research will focus on refining metrics for the "Innovation" component and applying the Service and Value IQ models to emerging sectors like autonomous transport and smart home appliances.

Summary updated

Abstract

The rapid development of artificial intelligence has brought the artificial intelligence threat theory as well as the problem about how to evaluate the intelligence level of intelligent products. Both need to find a quantitative method to evaluate the intelligence level of intelligence systems, including human intelligence. Based on the standard intelligence system and the extended Von Neumann architecture, this paper proposes General IQ, Service IQ and Value IQ evaluation methods for intelligence systems, depending on different evaluation purposes. Among them, the General IQ of intelligence systems is to answer the question of whether the artificial intelligence can surpass the human intelligence, which is reflected in putting the intelligence systems on an equal status and conducting the unified evaluation. The Service IQ and Value IQ of intelligence systems are used to answer the question of how the intelligent products can better serve the human, reflecting the intelligence and required cost of each intelligence system as a product in the process of serving human.
BibTeX
@misc{liu2017three,
  author = {Feng Liu and Yong Shi and Ying Liu},
  title = {{Three IQs of AI Systems and their Testing Methods}},
  year = {2017},
  eprint = {1712.06440},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1712.06440},
}
Figure