X-ray ptychography builds a nanoscale image by scanning a beam across a sample and recording thousands of overlapping diffraction patterns. Turning those patterns back into an image means solving a large nonlinear inverse problem by repeated gradient-style updates. At modern synchrotrons the detectors produce data faster than one GPU can hold or process, so solvers split the work across GPUs. The fastest published multi-GPU solver, PtyGer, still asks every GPU to agree on a single step size at every iteration. It finds that step with a line search: it tries candidate step lengths, runs the physics model forward over all diffraction patterns for each candidate, and combines the results across all GPUs. That step is both extra computation and a global synchronization point.
A separate line of optimization research shows that a good step size can be read off from quantities the solver already has. Taking the ratio of how far the iterate moved to how much the gradient changed gives a local estimate of curvature, with no trial evaluations and no proof of descent required. We propose to give each GPU its own step size for the image region it owns, estimated from that region's own gradient differences. A light neighbor-to-neighbor agreement on overlapping pixels replaces the global line search. The bet is that this removes both the extra forward passes and the global barrier while keeping PtyGer's reconstruction quality. Because local curvature in ptychography varies strongly with illumination and scan overlap, the per-region steps should also move faster than any single global step.
Core Claim
Scalable and accurate multi-GPU-based image reconstruction of large-scale ptychography data splits each conjugate-gradient iteration into four stages: local gradients, a master-GPU gather to form a search direction, a global line search, and local patch updates. It reports 1.6x scaling from 2 to 4 GPUs, against 1.3x for other tools, and attributes the shortfall from ideal partly to the line-search stage. Adaptive Gradient Descent without Descent (AdGD) shows that the step
$$\lambda_k=\min\Big\{\sqrt{1+\theta_{k-1}}\,\lambda_{k-1},\ \frac{\|x_k-x_{k-1}\|}{2\|\nabla f(x_k)-\nabla f(x_{k-1})\|}\Big\},\qquad \theta_{k-1}=\lambda_{k-1}/\lambda_{k-2},$$
converges for convex problems without function evaluations. It was about 2x faster than Armijo backtracking on logistic regression because it skips trial evaluations.
Our claim is that a block version of this rule works for domain-decomposed, nonconvex ptychography. In the block version, $x^{(b)}$ and $\nabla^{(b)} f$ are the pixels owned by GPU $b$ and the gradient restricted to them, and each block keeps its own $\lambda^{(b)}_k$. A step cap exchanged only between GPUs whose regions overlap keeps the blocks consistent. The resulting solver should reach PtyGer's reported quality (SSIM about 1.0, PSNR above 80 dB) in less wall-clock time and with strong scaling closer to linear.
The closest relative is Adaptive Federated Learning with Auto-Tuned Clients (Δ-SGD), which gives each federated client its own gradient-difference step. Its clients each hold a full copy of the model and are averaged afterward. Our blocks are disjoint, coupled pieces of a single image, so the steps act like a time-varying, block-diagonal rescaling of the gradient. That is precisely the coordinate-wise adaptivity the local-smoothness literature lists as missing.
Research Program
The work proceeds in three stages.
Stage 1: the solver. We will build the block-local solver on a PtyGer-style CUDA pipeline. Each GPU owns an image patch plus a halo of overlapping pixels. It computes its patch gradient from its own diffraction patterns plus halo contributions received point-to-point from neighbors, which removes the master gather.
It then sets its step with the block AdGD rule. Each GPU also exchanges its tentative step with its neighbors and takes the minimum, in the spirit of the neighbor-gossip variant of Adaptive Decentralized Composite Optimization via Three-Operator Splitting. That paper (DATOS) also lets agents set steps locally, but it finds them by local backtracking, which tests a descent inequality and therefore needs extra objective evaluations. It was tested only on synthetic machine-learning problems. We keep its consensus idea and drop the backtracking.
We will build two variants:
- Global variant. A single step from a two-scalar all-reduce of $\|x_k-x_{k-1}\|^2$ and $\|\nabla f_k-\nabla f_{k-1}\|^2$. This costs one tiny reduction and no forward passes.
- Hybrid variant. Keeps PtyGer's conjugate search direction but takes the AdGD step. This isolates whether conjugacy matters once the step is inexact.
Stage 2: streaming during the scan. Diffraction patterns arrive as the beam moves, so a block's objective changes between iterations. A naive gradient difference would then mix a change in geometry with a change in data. That is the failure mode Stochastic Adaptive Gradient Descent Without Descent fixes by evaluating both gradients on the same mini-batch, at the price of a second gradient evaluation.
Ptychography keeps all its patterns in GPU memory, so the same-batch difference comes for free here. The gradient on the patterns already present at step $k-1$ was computed then. At step $k$ we accumulate contributions from old and new patterns separately before summing them. This lets reconstruction start while the scan is still running, without a second pass.
Stage 3: analysis. We will try to extend AdGD's energy-function argument to block-diagonal, time-varying steps with neighbor caps, for smooth convex objectives that are coupled through overlap. We will check the result numerically on a convex, domain-decomposed least-squares problem before relying on it.
For the nonconvex ptychography objective we will not claim a proof. The AdGD paper itself reports only empirical success on nonconvex problems. We instead measure stability directly: divergence rate across random starting images and across initial steps spanning several orders of magnitude.
We note one tension openly. PtyGer insists on exact equivalence with sequential conjugate gradient, for stability. Our method gives that up. Whether this is acceptable is an empirical question, and we test it against a converged reference reconstruction.
Baselines
Fixed step (naive reference). Gradient descent with a step from a power-iteration estimate of the largest curvature at the initial iterate, safeguarded downward on divergence. It has zero synchronization and shows what any adaptivity has to beat.
PtyGer (strongest existing method). The conjugate-gradient solver with global line search, as described in Scalable and accurate multi-GPU-based image reconstruction of large-scale ptychography data. It is the strongest competitor because it was built for this exact data, beats PtychoPy and PtychoShelves, and reports:
- SSIM converging to about 1.0 and PSNR above 80 dB;
- 1.6x scaling from 2 to 4 GPUs;
- a 41.5x speedup of 8 GPUs over a memory-starved 4-GPU setup on the 16K Siemens-star data.
There is no reported result on Lacuna for PtyGer scaling across multiple nodes. We will run PtyGer ourselves on identical hardware, datasets, starting images and stopping criteria. Beating it means three things together:
- reaching its final PSNR/SSIM to within 0.5 dB and 0.001;
- doing so in lower wall-clock time on every dataset size;
- strong-scaling efficiency from 2 to 4 GPUs clearly above the 1.6x it reports, sustained to 8 GPUs and across two nodes.
Armijo backtracking. Gradient descent with backtracking on the global objective. This is the textbook adaptive step.
Ablations isolating the key ingredient. We compare global AdGD, block-local AdGD with no neighbor cap, and block-local AdGD with the cap. Together they separate three possible sources of gain: removing the line search, local curvature adaptivity, and the consensus safeguard.
All methods share the same initial step, or the same tuning budget of five values for any method that has a tuned rate. Every method is charged for all forward and adjoint passes and all communication.
Why not just…?
Why not just keep the line search and hide its latency by overlapping communication with computation? This is a real threat. The global reduction in a line search moves only scalars, so its cost may be latency and extra forward passes rather than bandwidth. Our first experiment profiles PtyGer's per-stage time on 2, 4 and 8 GPUs and on two nodes.
If the line-search stage turns out to be a small fraction of the iteration, communication overlap will capture most of the communication benefit. In that case our contribution shrinks to fewer forward passes plus faster progress per iteration from local steps. We will report it that way.
Why not just use the standard ptychography preconditioner that scales each pixel's step by the inverse local probe intensity? This fixed per-pixel bound already captures much of the spatial variation in curvature and needs no synchronization at all. It is the most likely way to make our block adaptivity redundant.
We will run it as a baseline in its own right. We will also apply the block AdGD rule on top of the preconditioned gradient. If block-local steps add nothing over the preconditioner alone, the block-wise part of the claim fails, and only the global line-search-free variant survives.
Why not just add GPUs? PtyGer's own results show that adding hardware helps mainly by escaping the memory wall: its 41.5x gain came from eliminating Unified Memory page faults. Scaling past that point was sublinear. Our scaling curves to 8 GPUs and two nodes test directly whether algorithmic synchronization, rather than raw compute, is the limiter.
Why not just use Adam or Barzilai–Borwein steps? Barzilai–Borwein steps use the same displacement-to-gradient-change ratio, but they are not safeguarded against step growth, and they are known to be erratic on nonconvex problems. AdGD's first term in the step rule is exactly that safeguard. Δ-SGD reported oscillating trajectories for Adam under heterogeneous data. Ptychographic blocks with very different illumination are heterogeneous in a similar way, but that is an analogy, not evidence. So we run both Barzilai–Borwein and Adam with the same tuning budget rather than assume they lose.
Proposed Experiments
Data. We use the APS Siemens-star, coins and pillar datasets at 4K, 8K and 16K diffraction patterns from the PtyGer evaluation. If they are not public, we use simulated scans of identical size and overlap.
Metrics. We report:
- wall-clock time to reach 0.99 of the reference SSIM and the reference PSNR minus 0.5 dB;
- the number of forward passes;
- the fraction of time spent in communication;
- strong-scaling efficiency;
- the fraction of 20 random starts that diverge, swept over initial steps from $10^{-4}$ to $10^{0}$ times the power-iteration step.
Decisive check 1: is the line search the bottleneck? The stage-time profile of PtyGer settles this.
Decisive check 2: do block-local steps beat the probe-intensity preconditioner at equal synchronization cost? If they do not, the block-wise part of the claim fails.
Decisive check 3: does the streaming same-batch correction improve on the naive difference? We start reconstruction when the scan is 25% complete and measure the image quality available at scan end.
Decisive check 4: is the convex block analysis predictive? On the convex domain-decomposed least-squares test, the neighbor cap should be necessary for convergence exactly when the analysis says it is.
References
- Scalable and accurate multi-GPU-based image reconstruction of large-scale ptychography data
- Adaptive Gradient Descent without Descent
- Adaptive Federated Learning with Auto-Tuned Clients
- Adaptive Decentralized Composite Optimization via Three-Operator Splitting
- Stochastic Adaptive Gradient Descent Without Descent