Abstract
Ptychography has rapidly grown in the fields of X-ray and electron imaging for its unprecedented ability to achieve nano or atomic scale resolution while simultaneously retrieving chemical or magnetic information from a sample. A ptychographic reconstruction is achieved by means of solving a complex inverse problem that imposes constraints both on the acquisition and on the analysis of the data, which typically precludes real-time imaging due to computational cost involved in solving this inverse problem. In this work we propose PtychoNN, a novel approach to solve the ptychography reconstruction problem based on deep convolutional neural networks. We demonstrate how the proposed method can be used to predict real-space structure and phase at each scan point solely from the corresponding far-field diffraction data. The presented results demonstrate how PtychoNN can effectively be used on experimental data, being able to generate high quality reconstructions of a sample up to hundreds of times faster than state-of-the-art ptychography reconstruction solutions once trained. By surpassing the typical constraints of iterative model-based methods, we can significantly relax the data acquisition sampling conditions and produce equally satisfactory reconstructions. Besides drastically accelerating acquisition and analysis, this capability can enable new imaging scenarios that were not possible before, in cases of dose sensitive, dynamic and extremely voluminous samples.
Introduction
Ptychography is a powerful imaging technique used with X-rays and electrons to achieve sub-angstrom resolution. It works by scanning a coherent beam across a sample and recording the resulting far-field diffraction patterns. However, detectors only record the intensity of the light, losing the phase information. Recovering this information traditionally requires solving a complex "phase retrieval" inverse problem using iterative, model-based algorithms like ePIE. These methods are computationally intensive and require a high degree of redundancy—adjacent scan points must typically overlap by at least 50%, limiting scanning speed and increasing radiation dose.
The PtychoNN Architecture
To overcome these limitations, the authors introduce PtychoNN, a deep convolutional neural network (CNN) designed to solve the phase retrieval problem end-to-end. Unlike iterative methods that require hundreds of passes through the data, PtychoNN learns the underlying physics of the diffraction-to-object mapping during training.
The architecture consists of three main components: an encoder arm that compresses input X-ray scattering data into high-level features, and two decoder arms. One decoder is dedicated to predicting the amplitude (structure) of the object, while the other predicts the phase. Once trained, PtychoNN can predict the object's properties for a single scan point in approximately 1 millisecond.
Breaking the Overlap Constraint
A significant advantage of PtychoNN is its ability to bypass the traditional 50% overlap constraint. Because the network learns to extract information from the "tails" of the beam, it can produce accurate reconstructions even with sparse sampling.
As shown in the experimental results, PtychoNN successfully reconstructs images with a 150 nm step size—a 5x reduction in sampling density compared to the standard 30 nm step. This translates directly to faster data acquisition and lower radiation damage for sensitive samples.
Key Results and Data Efficiency
Evaluated using experimental data from the Advanced Photon Source (APS), PtychoNN demonstrated a massive speedup, performing up to 300 times faster than high-performance iterative solvers like Ptycholib. Furthermore, the model is remarkably robust to limited data; it can produce high-quality results with as few as 800 training samples. This efficiency makes "on-the-fly" training during an experiment a realistic possibility, enabling true real-time feedback.
Conclusion
PtychoNN represents a breakthrough for real-time ptychographic imaging. By shifting the computational burden from iterative solvers to a pre-trained neural network, researchers can achieve high-resolution imaging with significantly reduced radiation doses and near-instantaneous reconstruction. This capability opens new doors for studying dose-sensitive, dynamic, and voluminous samples that were previously inaccessible to ptychography.
Summary updated