3D Reconstruction from Unposed Two Dimensional Projections

This direction is split into other directions: unknown particle orientation. Its last version stays readable and citable here.

Version 1 of 150 papers57 conceptsfrozen 2026-10-04

Imagine trying to determine the shape of a complex sculpture by looking only at its shadows cast on a wall. This would be simple if you knew exactly how the sculpture was being rotated between each snapshot. But in structural biology—specifically in Cryo-Electron Microscopy (cryo-EM) and X-ray Free-Electron Laser (XFEL) imaging—we are presented with a much harder version of this puzzle: we have millions of "shadows" (2D projections), but the orientation of the object for each snapshot is completely unknown.

This "orientation problem" is the central hurdle in reconstructing the 3D structures of proteins and viruses. Because these particles are suspended in liquid and captured mid-tumble, researchers must mathematically align thousands of noisy, random snapshots into a coherent 3D volume without any metadata about their spatial positioning.

The Iterative Approach: Expansion-Maximization-Compression

The most intuitive way to solve this is to guess and check. Veit Elser formalized this in the EMC (Expansion-Maximization-Compression) algorithm, described in Reconstruction algorithm for single-particle diffraction imaging experiments.

The logic follows a loop:

  1. Expansion: Start with a trial 3D model and generate all possible 2D projections (orientations).
  2. Maximization: Compare each experimental 2D snapshot against these simulated projections to find the most likely orientation.
  3. Compression: Update the 3D model by merging the snapshots according to their assigned orientations.

While powerful, this iterative process is computationally expensive and prone to getting stuck in "local optima"—incorrect 3D shapes that happen to explain the 2D data reasonably well.

Bypassing Poses: The Method of Moments

A more radical approach is to stop trying to find the orientation of every single particle. Instead, we can look at the global statistics of the entire dataset.

In Method of moments for 3-D single particle ab initio modeling with non-uniform distribution of viewing angles, Joe Kileel, Amit Singer, and colleagues demonstrate that the statistical moments (the mean and covariance) of the 2D images are mathematically linked to the 3D structure. If you have enough random views, the "average" of those views and the "variance" between them contain enough information to solve for the 3D volume directly.

Surprisingly, Kileel showed that having a non-uniform distribution of angles—where some views appear more often than others—actually makes the math easier by breaking symmetries that otherwise make the problem underdetermined. Further work by Tamir Bendory in Autocorrelation analysis for cryo-EM with sparsity constraints proved that if we assume the molecule is "sparse" (made of discrete atoms), we can reconstruct the structure with significantly fewer images than previously thought.

The Geometry of Projections: Manifold Learning

Another way to think about this problem is geometric. Every 2D projection of a 3D object is a point in a high-dimensional space. Because these projections all come from the same object, they must lie on a low-dimensional "manifold"—a curved surface representing all possible rotations.

Yi Fang and Karthik Ramani explored this using Spherical Multidimensional Scaling (sMDS) in Estimating view parameters from random projections for Tomography, treating projections as vectors and embedding them onto a sphere to recover their relative angles.

When the data is extremely noisy, measuring the distance between two projections becomes difficult. Amit Moscovich and Nathan Zelesko proposed using the Earthmover’s Distance (EMD) in Earthmover-Based Manifold Learning for Analyzing Molecular Conformation Spaces. EMD measures the "work" required to transform one image into another, which is far more robust to noise and small shifts than standard pixel-wise comparisons.

Neural Implicit Representations

The latest shift in the field moves away from discrete 3D grids (voxels) toward continuous neural functions. Instead of storing a 3D density map, we train a neural network to represent the volume.

In Reconstructing continuous distributions of 3D protein structure from cryo-EM images, Ellen D. Zhong and Joseph H. Davis introduced cryoDRGN. This framework uses a Variational Autoencoder (VAE) to learn not just a single 3D structure, but a continuous "landscape" of structures. This allows researchers to see how proteins move and change shape (conformations), solving the orientation problem and the heterogeneity problem simultaneously.

Similarly, Alireza Nasiri and Tristan Bepler developed TARGET-VAE, which uses group-equivariant neural networks to ensure that the model's understanding of an object is naturally invariant to its rotation and translation in the 2D image.

Why It Works

The reason we can reconstruct 3D from unposed 2D data is rooted in the Fourier Slice Theorem: each 2D projection is a central slice through the 3D Fourier transform of the object. Even if we don't know the angle of a slice, we know it must pass through the origin. By finding where these slices intersect in Fourier space, we can "lock" them into their correct relative orientations, effectively building the 3D structure from the inside out. Whether through iterative refinement, statistical moments, or deep learning, the goal remains the same: finding the consistent 3D volume that best explains the chaotic collection of 2D shadows.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_3d_reconstruction_from_unposed_two_dimen_930e0f96,
  title = {3D Reconstruction from Unposed Two Dimensional Projections},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/3d-reconstruction-from-unposed-two-dimensional-projections/rdr_930e0f96977f4407aebd1e0138b786af},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_3d_reconstruction_from_unposed_two_dimen_930e0f96_v1,
  title = {3D Reconstruction from Unposed Two Dimensional Projections},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/3d-reconstruction-from-unposed-two-dimensional-projections/rdr_930e0f96977f4407aebd1e0138b786af/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}