This document is a collection of keynote abstracts from the 2023 33rd International Conference on Field-Programmable Logic and Applications (FPL). Rather than a single research paper, it presents a high-level vision from industry and academic leaders (Stanford, Intel, MIT, CERN, and AMD) on the future of reconfigurable computing.
TL;DR
The FPL 2023 keynotes outline a pivot toward Reconfigurable Dataflow Accelerators (RDAs) and Adaptive SoCs to meet the "petaFLOPS-scale" demands of foundation models and exascale scientific data. The central theme is bridging the gap between high-level algorithms and hardware-centric design to handle increasingly irregular, data-intensive workloads.
What Problem Does This Paper Address?
The collective abstracts address the "computational wall" hit by traditional architectures when facing three major shifts:
- The Foundation Model Era: Models like GPT-3 require hundreds of petaFLOPS for training and inference. As these models evolve, they are becoming more "irregular," featuring sparse data access and complex control flows that traditional GPUs struggle to handle efficiently.
- The Data Deluge: Projects like CERN’s High-Luminosity LHC (HL-LHC) will soon generate an exabyte of data annually, requiring a "green revolution" in computing to process data without an unsustainable carbon footprint.
- The Programmability Gap: There remains a significant barrier between algorithm-centric developers (who use Python/PyTorch) and hardware-centric FPGA design.
Methods
The keynotes propose several novel architectural and software approaches:
- Reconfigurable Dataflow Accelerators (RDAs): Kunle Olukotun (Stanford) describes RDAs that exploit hierarchical dataflow parallelism. A new execution model called "Dataflow Threads" is introduced to specifically accelerate irregular applications.
- Computing Near Storage: Arvind (MIT) proposes building low-power appliances using flash storage and hardware accelerators. By processing data directly in an SSD form factor, these appliances aim to be 10x cheaper than traditional server clusters for analyzing terabyte-size, sparse graphs.
- Adaptive Architectures: Trevor Bauer (AMD) discusses the XDNA architecture, which integrates Xilinx’s FPGA and Adaptive SoC innovations into mainstream AMD portfolios to create a scalable platform for heterogeneous compute.
- Advanced EDA Tools: Nabeel Shirazi (Intel) emphasizes the need for next-generation EDA (Electronic Design Automation) software to bridge the gap between high-level algorithms and silicon, citing tools like Vivado IP Integrator and Model Composer as precursors.
Datasets
While specific training sets are not the focus of these abstracts, the authors reference the massive scales at which their systems must operate:
- Foundation Models: Massive unlabeled datasets used for training models with tens of billions of parameters.
- High-Energy Physics: The HL-LHC detector data, estimated at one exabyte per year.
- Graph Analytics: Terabyte-size, irregular, and sparse graphs used in social media and sensor data analysis.
Key Insights and Ideas
- Sparsity is the Future: Future AI models will move away from dense computations toward irregular data access. RDAs are positioned as the solution because they minimize off-chip communication by keeping dataflow local and reconfigurable.
- The "SSD Appliance" Concept: Instead of a 32-node server cluster, complex graph analysis could be performed on a specialized appliance that plugs into a laptop, drastically lowering the barrier to entry for big data analytics.
- Hardware-Software Synergy: The integration of Xilinx into AMD and the focus on Intel’s Quartus software signal that the future of FPGAs is not as standalone chips, but as "adaptive" components deeply integrated into broader SoC ecosystems.
- Sustainability as a Constraint: CERN’s Maria Girone highlights that exascale computing is no longer just about speed; it must be "green," focusing on carbon footprint reduction alongside performance.
Key Results
- Cost Efficiency: "Near-storage" appliances are projected to be 10x cheaper than 16-32 node server clusters for specific data-intensive tasks.
- Compute Requirements: The abstracts quantify the current state-of-the-art as requiring 100s of petaFLOPS for foundation model training.
- Patent/Industry Impact: The speakers represent a massive footprint in the field, with Trevor Bauer (AMD) noting over 70 patents and Nabeel Shirazi (Intel) noting 43, underscoring the commercial maturity of these "adaptive" technologies.
Limitations and Future Work
- The "Gap" Remains: Despite improvements in EDA tools, the transition from an algorithm-centric world to a hardware-centric world is still cited as a "major obstacle" for the next decade.
- Complexity of Irregular Flow: While "Dataflow Threads" are proposed for irregular applications, the abstracts acknowledge that future foundation models will have even longer sequence lengths and more complex architectures, suggesting a moving target for hardware designers.
- Scaling EDA: Intel’s keynote specifically identifies "EDA software" as a grand challenge, implying that current tools are not yet sufficient for the next generation of silicon design.
Summary updated