Electron tomography recovers the 3D positions of individual atoms in a nanoparticle by tilting it in a microscope and recording a series of 2D projections. The best current solvers treat each atom as a small 3D Gaussian blob and fit these blobs to each new specimen from scratch by gradient descent. This works, but it is slow. It also learns nothing from one sample that would help with the next, and it breaks down when real micrographs contain effects the simple projection model ignores, such as contamination, residual misalignment, and dose noise. We propose to replace that per-specimen fit with a learned solver. A trained network proposes atoms from the tilt series. It then refines them over a few recurrent steps. At each step it renders the current atoms into the measured tilt angles, compares the result against the real projections, and predicts corrections from the mismatch, with no gradient descent at test time. Training uses realistic simulated tilt series only.
The bet is that this measurement-feedback loop does two jobs at once. It removes per-specimen optimization cost, the classic weakness of neural 3D representations. It also narrows the synthetic-to-real gap, because every refinement step is anchored to the actual experimental data rather than to what the network learned from simulations. That matters for materials science, where structures are routinely mapped and then left without functional characterization. Structure determination that takes seconds and needs no expert supervision is a precondition for studying whole families of related compounds instead of single isolated ones.
Core Claim
Our claim is that a recurrent, measurement-driven atom solver can match per-specimen Gaussian optimization on in-distribution simulated tilt series at a small fraction of the runtime. It should also degrade less than both a single-pass network and per-specimen optimization when test data differ from training in noise, contamination, alignment error, or chemistry. The mechanism for the second part is specific. The solver renders atoms with a deliberately simple, cheap forward model: analytic line integrals of isotropic Gaussians. Training data, by contrast, come from a realistic multislice simulator. The learned update must therefore learn to read residuals that contain model mismatch, which a hand-written gradient step cannot do.
This combines three pieces of prior work. A Gaussian Parameterization for Direct Atomic Structure Identification in Electron Tomography models a specimen as spherical Gaussians, one per atom, with amplitude tracking atomic number. It fits them per specimen with an isotropy constraint and periodic merging of atoms that sit too close together. That paper supplies our representation and physical priors, and it shows those priors suppress missing-wedge streaks. It also leaves open the cost of optimizing from scratch and the lack of any prior shared across samples. ReSplat: Learning Recurrent Gaussian Splats supplies the refinement mechanism. It renders current Gaussians, computes error maps against input views, and predicts parameter updates through attention over nearest neighbours in four steps. It reports beating 4,000 iterations of per-scene splatting while running 100x faster, and a 5 dB PSNR gain over single-pass models at out-of-distribution resolutions. That second result is our main motivating evidence that feedback helps under distribution shift. ReSplat works only with pinhole RGB cameras and photometric losses. We move it to transmission projection physics, where the primitives are discrete, countable atoms whose existence must be decided, not per-pixel splats.
The closest competitor from the materials side, AutoMat, solves a related inverse problem. It uses a language-model agent that coordinates a denoiser, template retrieval, a crystal-file fitter and an interatomic potential, and it rolls back when checks fail. It takes about two minutes per case and is restricted to 2D materials from a single projection. Its authors call bulk 3D reconstruction from one projection ill-posed. We attack the 3D case, where a tilt series removes that ill-posedness, and we internalize the retry loop as a learned update instead of external tool calls.
Research Program
The solver has three parts.
The first is an encoder that takes the tilt series with known angles and produces a coarse 3D feature volume. We tokenize projections together with their ray geometry, analogous to the ray coordinates in GS-LRM, and lift them by backprojecting features. From this volume a head proposes an over-complete set of candidate atoms. Each candidate has a position, an amplitude, a width, and an existence score.
The second part is the recurrent loop, run for T steps with T around 4 to 8. At each step, surviving atoms are projected analytically into every tilt angle. The residual between measured and rendered projections is encoded, and each atom gathers residual features along its projection footprint in all views. A transformer restricted to spatially nearby atoms then predicts position and amplitude corrections plus an updated existence score. Singh et al.'s minimum-distance merge is applied as a fixed, non-learned operation between steps, so physical validity never depends on the network learning it.
The third part is a species head that classifies atomic number from the final amplitudes and local context.
Training data come from multislice simulation, the approach used via abTEM in Space-Time Implicit Neural Representations for Atomic Electron Tomography on Dynamic Samples. We simulate nanoparticles in the four structure classes Singh et al. study: periodic gold, grain-boundary iron-platinum, amorphous palladium, and high-entropy alloys. Tilt range is limited to ±70°. We vary dose, defocus and thickness. The loss matches predicted to true atoms with Hungarian assignment, penalizes position and species errors, and adds a projection-consistency term at each step. We deliberately hold out contamination, alignment jitter, one chemistry class, and particle sizes above the training range. Those held-out factors form the shift evaluation.
Baselines
The naive reference is the classical two-stage pipeline: filtered backprojection or the simultaneous algebraic reconstruction technique, followed by 3D peak finding. Singh et al. report that under a ±70° missing wedge both methods produce elongation streaks and many false-positive atoms. We run the standard implementations on identical tilt series and tune the peak-finding threshold on the validation split. It is the floor, and any learned method that fails to beat it clearly on false positives has failed.
The strongest existing method is per-specimen Gaussian optimization from A Gaussian Parameterization for Direct Atomic Structure Identification in Electron Tomography. It is the strongest competitor because it already uses the same representation and physical constraints and was validated on experimental data. We found no reported quantitative detection or position-error number on Lacuna for this method. The file notes only qualitative carbon–tellurium separation on the experimental carbon-nanotube sample. We will therefore reimplement it, tune its learning rate and merge schedule on validation, and report it at several iteration budgets, including one matched to our wall-clock time. Beating it means two things. The first is equal-or-better atom F1 and coordinate error in distribution at at least 10x lower runtime. The second is a strictly smaller drop in F1 under each held-out shift.
A single-pass version of our network, which is the proposal head with no recurrent steps, isolates the value of feedback. A second ablation, where the recurrent update receives only the atoms' own features and not the measured-minus-rendered residual, isolates whether gains come from anchoring to the measurement rather than from extra network depth. A third ablation renders with the multislice simulator inside the loop, which is too slow at test time but tells us how much the learned update compensates for forward-model mismatch.
For a secondary grounding check, the 2D special case can be run on STEM2Mat-Bench. There AutoMat reports 0.11 Å projected RMSD, 83.2% structure success and about two minutes per case, and baselines such as MicroscopyGPT reach 1.3–1.6 Å. This comparison is only indicative, since that benchmark is single-projection 2D rather than tomography.
Why not just…?
Why not just run per-specimen Gaussian optimization and accept the runtime? For one sample a few minutes is tolerable. It stops being tolerable for time-resolved series, for screening libraries of related compounds, and for feedback during acquisition. Speed alone would make this an engineering paper, however. The scientific question is whether a learned prior plus measurement feedback is more robust than fitting from scratch. From-scratch fitting has no knowledge of what nanoparticles look like, so under contamination and low dose it can fit artifacts as atoms. The matched-runtime and shift experiments test exactly this. If per-specimen optimization is equally robust, we will report that the method's value is speed only.
Why not just predict atoms in one pass and then run a few gradient steps of the per-specimen optimizer? This hybrid is the most serious threat, and we include it as a baseline at equal wall-clock time. If it matches the recurrent solver in and out of distribution, then the learned update adds nothing beyond a good initialization, and we will say so. Our reason to expect otherwise is that hand-written gradients of a simple Gaussian projection model cannot account for the mismatch between that model and multislice physics or contamination. A learned update trained on realistic simulations can.
Why not just train a direct network on heavily augmented simulations and skip the feedback? The single-pass ablation is exactly this model, trained on the same augmentations. Evidence that direct regression is fragile comes from other domains. MarrNet found direct image-to-3D regression at 0.52 IoU against 0.57 with an intermediate representation. Stress-Testing Multimodal Foundation Models for Crystallographic Reasoning found unconstrained models' errors grow by orders of magnitude under compositional shift. The held-out chemistry split reproduces that stress in our setting.
Why not just denoise the projections and then reconstruct classically? Denoising does not remove the missing wedge, which is the source of the streaks and false atoms Singh et al. document. We include a denoiser-plus-backprojection variant to confirm this.
Proposed Experiments
The primary benchmark is a held-out simulated test set in the four structure classes. Metrics are atom detection precision, recall and F1 at a 0.5 Å matching tolerance, root-mean-square position error of matched atoms, species accuracy, and runtime on one GPU. Shift splits each add one held-out factor: dynamic carbon contamination modelled as in the space-time study, angular jitter, dose reduced below the training range, a held-out alloy composition, and particles larger than any training particle.
The decisive checks are these. First, in distribution, the recurrent solver's F1 should come within a few points of converged per-specimen optimization at at least 10x lower runtime. Second, on every shift split its F1 drop should be smaller than both per-specimen optimization and the single-pass model. Third, the residual-free ablation should lose most of that robustness advantage. Fourth, on the experimental zirconium–tellurium carbon-nanotube dataset, where no ground truth exists, the recovered structure should agree with per-specimen optimization and should separate carbon from tellurium by amplitude. Its projection residuals on angles held out from the input should be no larger than the baseline's.
If the second or third check fails, the claim that feedback bridges the synthetic-to-real gap is wrong, and the contribution reduces to an amortized solver. The hybrid baseline then decides whether even that is worth having.
References
- A Gaussian Parameterization for Direct Atomic Structure Identification in Electron Tomography
- ReSplat: Learning Recurrent Gaussian Splats
- AutoMat: Physics-Guided Agentic Reasoning for Solving Ill-Posed Inverse Microscopy Problems
- GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting
- Space-Time Implicit Neural Representations for Atomic Electron Tomography on Dynamic Samples
- MarrNet: 3D Shape Reconstruction via 2.5D Sketches
- Stress-Testing Multimodal Foundation Models for Crystallographic Reasoning