FUTURE TRENDS IN COMPUTER HARDWARE

E. A. Feustel, Christian S. Jensen, Fiona Mcmahon1973 · DOI

TL;DR

This paper explores the future of computer hardware for scientific computation, arguing that while vector and parallel processors offer immense speedups, even small amounts of scalar code can drastically reduce performance. The authors propose that a more "unstructured" machine architecture, combined with a generalized "vector" concept that includes code segments, could lead to more balanced and efficient high-performance computing.

What Problem Does This Paper Address?

As computational physics problems grow in complexity, so does the demand for faster computing. The paper grapples with how to best achieve this speedup by examining different processor architectures: serial (SISD), single instruction stream-multiple data stream (SIMD, like parallel processors), and multiple instruction stream-multiple data stream (MIMD, like concurrent processors). The core challenge identified is the significant performance disparity between highly optimized vector/parallel code and residual scalar code. This gap means that even a small percentage of scalar operations can cripple the overall speed of a system designed for massive parallelism, making it difficult to achieve the theoretical maximum utility of these advanced architectures. The paper also touches on the difficulty of programming for these machines and the need for new programming paradigms.

Methods

The authors analyze the utility of different processor architectures by evaluating them against a set of real-world scientific codes used at Lawrence Livermore Laboratory (LLL). They categorize hardware into four types based on Flynn's taxonomy (SISD, SIMD, MISD, MIMD) and discuss the characteristics of codes that perform well on vector and parallel machines. A key aspect of their analysis is the concept of "utility," which they define not just by peak speed but also by the range of numerical techniques a processor can comfortably handle. They also introduce the idea of "stack loops" as a way to emulate vector instructions on serial processors, effectively creating a more balanced architecture. The paper emphasizes the importance of considering the entire program, not just inner loops, when assessing performance.

Datasets

Not discussed in the paper. The paper uses a set of "real codes" from Lawrence Livermore Laboratory for evaluation, but these are not presented as formal datasets.

Key Insights and Ideas

  • The Scalar Bottleneck: Even a small fraction of scalar code can disproportionately degrade the performance of highly parallel or vector processors due to the massive difference in execution speeds.
  • "Utility" Beyond Peak Speed: A processor's true usefulness is determined by its ability to handle a broad range of numerical techniques and its overall program efficiency, not just its theoretical maximum speed.
  • "Stack Loops" as a Bridge: Emulating vector instructions using "stack loops" on concurrent processors can create a more balanced architecture, offering good scalar performance while enabling efficient vector operations.
  • The Power of "Unstructured" Machines: The authors advocate for less rigidly structured hardware, suggesting that flexibility in how functional units and memory are accessed can lead to better overall performance, especially when dealing with diverse computational tasks.
  • Generalized "Vector" Concept: The idea of a "vector" should be extended beyond simple numerical arrays to include code segments, allowing for more flexible and efficient programming of complex operations.
  • Bit Control Vectors are Crucial: For mapping and conditional operations, bit control vectors are presented as a more powerful and flexible alternative to traditional indexing, especially for irregular data structures.

Key Results

  • The paper highlights that even 10% scalar code running at 10% of vector speed can reduce overall performance by half (M ≈ 0.5).
  • Monte Carlo simulations and implicit differencing schemes are identified as areas where vector and parallel machines may struggle due to shorter vectors and complex dependencies, potentially leading to significant start-up time penalties.
  • The authors suggest that for vector machines, vectors need to be several hundred elements long to keep start-up time penalties below 10% of total operation time.
  • The "vector 7600" emulation (using stack loops on a CDC 7600) achieved speeds of 8-11 floating-point operations per microsecond for dyadic and triadic operations, respectively, which is a significant improvement over standard serial execution.

Limitations and Future Work

  • Programming Difficulty: The paper acknowledges that programming for vector and parallel machines is challenging and requires a significant shift in thinking from traditional scalar programming. It's unlikely that simply recompiling existing programs will yield good performance.
  • Compiler Limitations: Current compilers struggle to generate efficient vector code from high-level languages, especially for complex scenarios.
  • Implicit Differencing: A robust solution for handling full implicit differencing schemes on vector processors remains elusive.
  • Future Work: The authors suggest exploring architectures that generalize the concept of a "vector" to include code segments and advocate for more "unstructured" machine designs. They also point to the potential of tagged architectures for improved memory management and flexibility. The development of more sophisticated programming approaches that can adapt to various processor types is also a key area for future research.

Summary updated

Abstract

Vector processors, parallel processors, and sequential processors with much concurrency are considered. Some real codes can be shown to be almost completely suitable for parallel and vector processors . However, vector to scalar speed ratios can be so large for the vector and parallel machines that even a small residual scalar content is a serious problem. Monte Carlo approaches and implicit differencing schemes yield shorter vectors than explicit differencing schemes, causing problems due to long vector operation start up t imes. Programming in a vector language may be a more global and easier to understand approach to programming. It is necessary to use such an approach in order to attain some reasonable fraction of the potential speed of a vector or parallel processor. Introduction and Summary The utility of various scientific processor architectures is weighed against a set of codes used at Lawrence Livermore Laboratory (LLL). On the basis of experience gained in the development of parallel programs, vector machines, parallel machines, and concurrent machines are considered. The idea of utility is interpreted to relate to the question of how useful the maximum speeds of processors are when an entire program, rather than small parts, such as inner loops, is the weighting factor. Utility is also associated with the range of classes of numerical techniques which can be comfortably and easily mated to a given processor architecture. It is desirable to consider ease of programming as an aspect of utility. It has not been possible to consider this as quantitatively as the other questions. But some comments are made on the subject. The future may bring better understanding of general programming approaches simultaneously suited to many classes of processor. For this paper, it is usually assumed that coding is done in an appropriate assembly language, or in a language especially well mated to a given architecture. The point of view has been adopted that processing speed in computational physics is closely related to the average number of floating point operations done per unit of time. It is recognized that this approach is not entirely correct, but it is better than relating speed to instructions executed per unit t ime. Hardware designers face common technological l imits. Various design approaches have been the result of the competition to achieve maximum atility within these limits. As a result, parallel processors, vector processors, and serial processors with instruction execution overlap (concurrency) have emerged as contenders for maximum scientific utility.
BibTeX
@misc{feustel1973future,
  author = {E. A. Feustel and Christian S. Jensen and Fiona Mcmahon},
  title = {{FUTURE TRENDS IN COMPUTER HARDWARE}},
  year = {1973},
  doi = {10.2514/6.1973-3000},
}
Figure