TL;DR
This paper presents a novel system for real-time tracking of plasma instabilities in tokamaks using high-speed camera data processed directly on an FPGA. By deploying a machine learning model on the camera's onboard FPGA, the system achieves microsecond-level latency, enabling immediate feedback for plasma control and opening doors for advanced diagnostics in fusion research and beyond.
What Problem Does This Paper Address?
Magnetic confinement fusion devices, like tokamaks, aim to harness the power of plasma. However, plasma can become unstable, threatening operation. To maintain stable conditions and achieve fusion-relevant parameters, active feedback control is crucial. This control requires rapid, non-invasive diagnostics that can detect instabilities and generate corrective signals within microseconds – a timescale dictated by the rapid evolution of plasma phenomena. Traditional methods often struggle to meet these stringent latency requirements, especially when dealing with the high data rates from modern optical diagnostics like high-speed cameras. The challenge lies in processing vast amounts of image data in real-time to extract critical information about plasma behavior and translate it into control actions with minimal delay.
Methods
The core innovation lies in deploying a machine learning model directly onto the Field-Programmable Gate Array (FPGA) of a high-speed camera's frame grabber. Specifically, the authors utilize a Convolutional Neural Network (CNN) trained to predict the amplitude and phase of the n=1 Magnetohydrodynamic (MHD) mode from camera images.
Key methodological aspects include:
- Hardware Acceleration: Instead of offloading processing to a separate computer, the CNN inference is performed in situ on the FPGA. This drastically reduces data transfer bottlenecks and associated latencies.
- High-Level Synthesis (HLS): The authors leverage the
hls4mltoolchain to translate their TensorFlow CNN model into hardware description language (HDL) suitable for FPGA implementation. This bridges the gap between machine learning frameworks and hardware design. - Model Optimization for Hardware: Significant effort was dedicated to optimizing the CNN for the resource-constrained FPGA. This involved:
- Quantization: Using arbitrary-precision fixed-point representations instead of full 32-bit floating-point numbers to reduce memory and computational requirements.
- Sparsity: Applying pruning techniques to set a significant percentage of model weights to zero, further reducing computational load.
- Reuse Factor Tuning: Adjusting the "reuse factor" for different layers to control the degree of parallelization and find a balance between latency and FPGA resource utilization.
- Data Flow Management: Custom firmware was developed to handle image stream reordering (as camera data can arrive in non-sequential "stripes") and to embed model predictions back into the image stream for validation.
Datasets
The CNN models were trained and evaluated using a dataset collected from previous experiments on the High Beta Tokamak-Extended Pulse (HBT-EP) device. This dataset comprised:
- Image Data: High-speed camera frames (250 kfps, 128x64 pixels, 12-bit grayscale) from 45 plasma discharges.
- Ground Truth Data: Sine and cosine components of the n=1 MHD mode, derived from in-vessel low-field-side magnetic sensor arrays (FB3) using Fourier decomposition.
The dataset was split into training (40 shots) and testing (5 shots) sets, with further random splitting of the training set for validation.
Key Insights and Ideas
- FPGA as a Real-time ML Inference Engine: The paper demonstrates that FPGAs, when coupled with appropriate HLS tools and optimization techniques, are highly capable of performing complex ML inference for real-time control applications, achieving latencies far beyond what is typically possible with CPUs or even GPUs for this specific task.
- Co-design is Crucial: The success hinges on a tight co-design loop between the ML model architecture, its hardware implementation, and the specific constraints of the target FPGA. Simply porting a model is insufficient; significant optimization is required.
- Optical Diagnostics for Control: Moving beyond traditional magnetic sensors, using high-speed optical camera data for real-time feedback control offers a non-invasive and flexible diagnostic approach, potentially simplifying system integration and maintenance.
- Microsecond-Level Latency is Achievable: The demonstrated 17.6 µs trigger-to-output latency is a significant achievement, directly addressing the need for control signals that can keep pace with fast plasma dynamics.
Key Results
- Low Latency: The system achieved a total trigger-to-output latency of 17.6 µs. This breaks down into:
- 3 µs for camera exposure
- 5.3 µs for camera readout and buffering
- 7.7 µs for CNN model inference
- 1.6 µs for serial writeout
- High Throughput: The system can operate at a throughput of up to 120 kfps, with a minimum sampling interval of 8.3 µs.
- Model Performance: The "Optimized" CNN model, while slightly reducing prediction accuracy compared to larger models, still significantly outperformed traditional non-deep-learning methods and met the stringent latency and resource requirements.
- Resource Efficiency: The optimized model achieved a significant reduction in FPGA resource usage (e.g., ~36.7% LUTs, ~10.9% Registers, ~7.8% DSPs, ~9.3% BRAMs), making it feasible for deployment on the constrained frame grabber FPGA.
Limitations and Future Work
- Single Mode Tracking: The current implementation is designed for tracking a single n=1 MHD mode. Future work will explore multi-mode control tasks, which would require more complex firmware and potentially alternative hardware configurations.
- Fixed Control Parameters: The gain and phase shift parameters for the control signals are currently fixed. Future iterations will need the ability to adjust these parameters dynamically for different control campaigns.
- Dataset Staleness: Recent upgrades to the HBT-EP device have altered plasma conditions, making the existing training dataset potentially obsolete. New data collection and model retraining will be necessary for future experiments.
- Closed-Loop Control: While the system is designed for closed-loop control, actual feedback experiments have not yet been conducted. This requires integrating additional DAC and breakout board systems.
- Domain Adaptation and Reinforcement Learning: The authors suggest exploring more advanced ML techniques like domain adaptation, active learning, and reinforcement learning for more robust and adaptive control systems, moving beyond supervised learning.
- Broader Applications: The demonstrated workflow and hardware platform are applicable to a wide range of scientific domains beyond fusion, including cell classification, electron diffraction, and object detection.
Summary updated