DOE Advanced Scientific Computing Advisory Subcommittee (ASCAC) Report: Top Ten Exascale Research Challenges

Robert F. Lucas, James Ang, Keren Bergman, Shekhar Borkar, William Carlson, Laura Carrington, George T.‐c. Chiu, Robert P. Colwell, William J. Dally, Jack Dongarra, Al Geist, Gary Los Alamos National Laboratory Grider, Rud Haring, Jeffrey Hittinger, Adolfy Hoisie, D. Klein, Peter M. Kogge, Richard Lethin, Vivek Sarkar, Robert Schreiber, John Shalf, Thomas Sterling, Rick Stevens, Jon Bashor, Ron Brightwell, P. Coteus, Erik Debenedictus, Jon Hiller, K. H. Kim, Harper Langston, James Sandia National Laboratories Laros, Sven Argonne National Laboratory Leyffer, Richard Micron Murphy2014 · DOI

Abstract

Exascale computing systems are essential for the scientific fields that will transform the 21st century global economy, including energy, biotechnology, nanotechnology, and materials science. Progress in these fields is predicated on the ability to perform advanced scientific and engineering simulations, and analyze the deluge of data. On July 29, 2013, ASCAC was charged by Patricia Dehmer, the Acting Director of the Office of Science, to assemble a subcommittee to provide advice on exascale computing. This subcommittee was directed to return a list of no more than ten technical approaches (hardware and software) that will enable the development of a system that achieves the Department's goals for exascale computing. Numerous reports over the past few years have documented the technical challenges and the non-viability of simply scaling existing computer designs to reach exascale. The technical challenges revolve around energy consumption, memory performance, resilience, extreme concurrency, and big data. Drawing from these reports and more recent experience, this ASCAC subcommittee has identified the top ten computing technology advancements that are critical to making a capable, economically viable, exascale system.

Introduction: The Looming Power Wall

In 2014, the high-performance computing (HPC) community faced a critical "power wall." While Moore’s Law continued to increase transistor counts, the end of Dennard scaling meant that power density was no longer constant. Simply scaling existing petascale architectures to exascale performance (10^18 operations per second) would have required gigawatts of electricity—equivalent to the output of several nuclear power plants.

Scientific simulation visualization illustrating the complexity and data intensity of exascale computing targets.

The ASCAC subcommittee addressed the fundamental question of how to achieve a 1,000x increase in usable performance while maintaining a strict 20-megawatt power budget. This requires a revolutionary shift in how systems are designed, moving away from general-purpose commercial trends toward specialized scientific requirements.

Methodology: Strategic Synthesis and Co-design

This report represents a high-level strategic synthesis rather than a traditional experimental study. The DOE assembled a subcommittee of 23 experts from industry leaders (Intel, NVIDIA, IBM, HP), academic institutions, and national laboratories to reach a consensus on the most critical barriers to exascale.

Subcommittee members and contributors who identified the top ten exascale research challenges.

The group analyzed historical data, such as TOP500 performance trends and CMOS energy projections, to identify the "tipping points" where current technologies fail. The central methodology proposed is Co-design: an iterative process where scientific application requirements directly influence hardware architecture and system software, ensuring that the resulting machines are optimized for complex simulations rather than general consumer tasks.

Key Technical Insights

The subcommittee identified several paradigm shifts necessary for exascale:

  • Energy as the Primary Constraint: Achieving the 20 MW goal requires a 10x improvement in energy efficiency beyond standard industry trajectories. Technologies like Near-Threshold Voltage (NTV) circuits offer promise but introduce significant performance variability that software must manage.
  • Data Movement Costs: The energy cost of moving data (to memory or across networks) now dwarfs the cost of computation. This necessitates "Communication-Avoiding" algorithms and hardware innovations like Silicon Photonics to break bandwidth barriers.
  • Extreme Concurrency: Exascale systems will involve billions of execution threads. Traditional programming models like MPI are insufficient for this scale, requiring new approaches to handle massive load imbalances and data locality.
  • Resilience and Silent Errors: At exascale, hardware faults become frequent events. The system must move beyond simple checkpoint/restart methods toward algorithmic-based fault tolerance and robust "silent error" detection.

The Top Ten Research Challenges

The report culminates in ten specific R&D pillars:

  1. Energy Efficiency: Advanced power management and NTV.
  2. Interconnect Technology: Photonics and hierarchical fabrics.
  3. Memory Technology: 3D stacked memory and non-volatile memory (NVM).
  4. Scalable System Software: Lightweight, low-noise operating systems.
  5. Programming Systems: Models expressing data locality and resilience.
  6. Data Management: In-situ analysis to bypass I/O bottlenecks.
  7. Exascale Algorithms: Minimizing synchronization and communication.
  8. Discovery/Design/Decision Algorithms: Integrating Uncertainty Quantification (UQ).
  9. Resilience and Correctness: Ensuring accuracy despite frequent faults.
  10. Scientific Productivity: Optimizing the human "time-to-solution."

Conclusion and Future Directions

The challenges identified are deeply interdependent; a breakthrough in hardware may necessitate a complete rewrite of the software stack. The subcommittee advocates for an Open Exascale System Design Framework to allow researchers to test proxy applications on simulated architectures years before hardware deployment. Without sustained, targeted government investment, the U.S. risks falling behind as commercial markets prioritize mobile and cloud computing over the extreme demands of scientific exascale systems.

Summary updated

Abstract

Exascale computing systems are essential for the scientific fields that will transform the 21st century global economy, including energy, biotechnology, nanotechnology, and materials science. Progress in these fields is predicated on the ability to perform advanced scientific and engineering simulations, and analyze the deluge of data. On July 29, 2013, ASCAC was charged by Patricia Dehmer, the Acting Director of the Office of Science, to assemble a subcommittee to provide advice on exascale computing. This subcommittee was directed to return a list of no more than ten technical approaches (hardware and software) that will enable the development of a system that achieves the Department's goals for exascale computing. Numerous reports over the past few years have documented the technical challenges and the non¬-viability of simply scaling existing computer designs to reach exascale. The technical challenges revolve around energy consumption, memory performance, resilience, extreme concurrency, and big data. Drawing from these reports and more recent experience, this ASCAC subcommittee has identified the top ten computing technology advancements that are critical to making a capable, economically viable, exascale system.
BibTeX
@misc{lucas2014doe,
  author = {Robert F. Lucas and James Ang and Keren Bergman and Shekhar Borkar and William Carlson and Laura Carrington and George T.‐C. Chiu and Robert P. Colwell and William J. Dally and Jack Dongarra and Al Geist and Gary Los Alamos National Laboratory Grider and Rud Haring and Jeffrey Hittinger and Adolfy Hoisie and D. Klein and Peter M. Kogge and Richard Lethin and Vivek Sarkar and Robert Schreiber and John Shalf and Thomas Sterling and Rick Stevens and Jon Bashor and Ron Brightwell and P. Coteus and Erik Debenedictus and Jon Hiller and K. H. Kim and Harper Langston and James Sandia National Laboratories Laros and Sven Argonne National Laboratory Leyffer and Richard Micron Murphy},
  title = {{DOE Advanced Scientific Computing Advisory Subcommittee (ASCAC) Report: Top Ten Exascale Research Challenges}},
  year = {2014},
  doi = {10.2172/1222713},
}
Figure