The impact of data flow computing thinking on the development of computer architecture

Chuliang Weng, Xiaoli Gong, Chen Xu, Huayou Su, Aoying Zhou, Ge Yu, Z H Li, Yong Dou, Kenli Li, Wangdong Yang, Jiangshan WangScientia 2020 · DOI

Abstract Throughout the history of computer architecture, the von Neumann model has been the dominant paradigm for system design. However, data flow computer systems represent the most extensively studied non-von Neumann architecture. This paper analyzes the critical role that "data flow computing thinking" has played in driving innovation and architectural evolution. We review the foundational principles of data flow computing and the limitations of its early implementations. We then examine how these principles were successfully integrated into modern CPUs through out-of-order execution and multi-threading. Furthermore, we explore the influence of data flow on stream processing, modern GPUs, and the current era of Big Data and AI. Finally, we summarize the governing rules of data flow thinking and forecast future development trends.

Introduction

For decades, the computer industry has been defined by the von Neumann architecture, characterized by sequential instruction execution and a clear separation between processing and memory. While successful, this model eventually encountered the "von Neumann bottleneck," where the speed of data transfer between the CPU and memory limits overall performance. Data flow computing emerged as a radical alternative, proposing that instructions should execute as soon as their required data (operands) become available, rather than following a rigid program counter. Although "pure" data flow machines struggled commercially, the underlying logic—data flow thinking—has become the invisible engine powering modern high-performance computing.

The Evolution of Dataflow Thinking

The history of data flow computing can be categorized into distinct eras, moving from experimental research to mainstream integration. Early pioneers in the 1970s and 80s developed static and dynamic data flow models to exploit fine-grained parallelism.

Evolutionary timeline of dataflow computing, from early research prototypes to modern GPUs.

As shown in the timeline above, the journey began with the MIT static dataflow computer in 1974, followed by dynamic models like the Monsoon in 1990. These early systems faced significant overhead in managing "tokens" (data packets) and required specialized programming languages, which hindered their adoption. However, by the turn of the millennium, the core concepts began to "infiltrate" mainstream architecture.

Dataflow in Modern CPUs: The Infiltration Era

Modern CPUs are often described as "von Neumann on the outside, data flow on the inside." To overcome the limitations of sequential execution, architects adopted two key technologies rooted in data flow thinking:

  1. Out-of-Order (OoO) Execution: Instead of waiting for a stalled instruction (e.g., a slow memory load), the CPU looks ahead in the instruction stream to find later instructions whose data is ready. It essentially builds a dynamic data flow graph on the fly to maximize hardware utilization.
  2. Simultaneous Multithreading (SMT/Hyper-threading): Introduced around 2000, this technology allows multiple threads to share execution resources. When one thread is blocked by a data dependency, the processor can switch to another thread whose data is ready, effectively hiding latency through data-driven scheduling.

Stream Processing and the GPU Revolution

The early 21st century saw the rise of stream processing, which treats data as a continuous flow through a series of computation kernels. Stanford’s Imagine processor was a notable research milestone that influenced the development of modern Graphics Processing Units (GPUs).

By 2006, with the introduction of architectures like NVIDIA's G80, GPUs moved beyond fixed-function graphics to become general-purpose parallel processors. By adopting a stream-based data flow model, GPUs can manage tens of thousands of concurrent threads, far exceeding the parallel capabilities of traditional CPUs for data-intensive tasks.

Dataflow in the Era of Big Data and AI

Today, data flow thinking has moved beyond hardware into the software and systems layer. The massive datasets of the modern era require architectures that prioritize data movement over control logic.

  • Software Frameworks: Distributed engines like Apache Spark and Flink utilize Directed Acyclic Graphs (DAGs) to represent computations. By focusing on how data flows between transformations (Map, Reduce, Filter), these frameworks can automatically optimize execution across thousands of servers.
  • AI Accelerators: Neural networks are inherently data flow graphs where tensors flow through layers of neurons. Specialized chips like the Google TPU and Cambricon (DianNao) achieve extreme efficiency by mapping these graphs directly onto hardware structures like systolic arrays, which are a form of rigid, high-performance data flow.

Conclusion and Future Outlook

Data flow computing thinking has proven to be a resilient and transformative force in computer architecture. While the "pure" data flow machines of the 1970s were ahead of their time, their principles now underpin every major breakthrough in performance, from instruction-level parallelism in CPUs to the massive throughput of AI accelerators.

Looking forward, the authors anticipate a move toward "Intelligent Dataflow"—architectures that can dynamically reconfigure their data paths based on the specific requirements of AI workloads. Bridging the gap between the ease of von Neumann programming and the efficiency of data flow execution remains the final frontier for the next generation of computer systems.

Summary updated

Abstract

Throughout the entire history of computer architecture, the von Neumann model has been the most mainstream model for computer systems architecture. Data flow computer systems are undoubtedly the most well-studied type of non-von Neumann computer architecture. In this paper, from the perspective of the computer architecture development, we analyze the important role that data flow computing thinking has played in both innovation in computer architecture and in the development of computer architecture over time. We first review the thinking behind data flow computing and analyze the limitations it faced in its early stages. Then we analyze two important technologies used in modern CPUs-out-of-order execution and multi-threading technology-that were both developed using data flow computing thinking. We continue to view streaming computing, streaming processor architecture, and modern GPUs through the lens of data flow computing thinking. Further, we analyze how to apply data flow computing thinking to the development of computer systems in the era of big data intelligence. Finally, we summarize the rules of data flow computing thinking and forecast future development trends.
BibTeX
@inproceedings{weng2020impact,
  author = {Chuliang Weng and Xiaoli GONG and Chen Xu and Huayou Su and Aoying Zhou and Ge Yu and Z H Li and Yong Dou and Kenli Li and Wangdong Yang and Jiangshan Wang},
  title = {{The impact of data flow computing thinking on the development of computer architecture}},
  booktitle = {Scientia},
  year = {2020},
  doi = {10.1360/ssi-2020-0092},
}
Figure