Learning from high-resolution 2D images to reconstruct 3D microstructures

Wenbo Sui, Lars P. Mikkelsen, Jens W. Andreasen, Daniel Lichau, Harold Phelippeau, Philip J. WithersarXiv 2026 · arXiv

Understanding the internal structure of materials—their microstructure—is the key to predicting how they will perform under stress or how well they will conduct heat. However, getting high-resolution 3D data is a massive bottleneck. You either need a multi-million dollar synchrotron, or you have to painstakingly slice and destroy your sample.

A new paper by Sui et al. proposes a clever workaround: Privileged Cross-modal Relational Distillation (PCRD). By using high-resolution 2D images (which are easy to get) to "teach" a model how to interpret low-resolution 3D scans, they’ve found a way to reconstruct specimen-specific 3D microstructures with unprecedented fidelity.

TL;DR

The researchers developed a framework that uses "privileged" high-resolution 2D SEM images during training to improve 3D reconstructions from low-resolution micro-CT scans. Their novel relational distillation method allows a student model to learn fine-scale spatial organization without needing the two imaging modalities to look identical at a feature level.

What Problem Does This Paper Address?

In materials science, we often face a trade-off. Laboratory micro-CT can scan large volumes quickly, but it lacks the resolution to clearly separate individual features, like densely packed carbon fibers. Conversely, Scanning Electron Microscopy (SEM) provides beautiful, high-resolution 2D detail but lacks 3D context.

Existing methods often try to synthesize "statistically representative" 3D volumes from 2D slices. While these look right, they aren't specimen-specific—they don't match the actual physical sample you're holding. The challenge is bridging the "modality gap" between the blurry, low-res CT and the sharp, high-res SEM to get a reconstruction that is both high-resolution and physically accurate to the specific specimen.

Methods

The authors frame this as a Learning Using Privileged Information (LUPI) problem. During training, the model has access to both the "privileged" high-res SEM and the low-res CT. At inference time (when you're actually using it), it only needs the CT.

Overview of the PCRD framework, showing the SEM teacher and CT student branches.

The secret sauce is Anchored Cross-modal Similarity Distribution Distillation (ACSD). Instead of trying to force the CT features to match the SEM features directly (which is hard because the physics of the two imaging methods are so different), they match the relationships between pixels.

  1. Anchors: They select specific spatial locations (anchors) in both images.
  2. Similarity Maps: For each anchor, they calculate how similar it is to every other pixel in its own modality.
  3. Distillation: The "student" (CT model) is trained to produce similarity distributions that match the "teacher" (SEM model). This transfers the high-res spatial organization from the SEM to the CT branch.

The architecture uses frozen DINOv3 ViT-B/16 encoders to extract features, which are then fused and passed to a convolutional decoder that predicts fiber instances using flow fields (similar to the popular Cellpose algorithm).

The ACSD mechanism: matching similarity distributions between anchors and the surrounding feature field.

Datasets

  • Primary Dataset: 3,200 spatially registered pairs of SEM and micro-CT images of pultruded carbon fiber-reinforced polymer (CFRP) composites.
  • Validation Dataset: An independent high-resolution synchrotron X-ray CT dataset of the same material, used as a "ground truth" to verify the 3D reconstruction quality.

Key Insights and Ideas

  • Relational > Direct: Matching internal relationships (how pixel A relates to pixel B) is far more effective for cross-modal learning than direct feature alignment. It bypasses the differences in imaging physics and contrast.
  • Anchor Density: You don't need to supervise every single pixel. The authors found that using a subset of anchors (around 2,592 for a 1152x1152 patch) actually performs better than dense supervision, likely by reducing redundant constraints.
  • Temperature Tuning: By using different "temperatures" for the teacher and student probability distributions, they could sharpen the teacher's target while allowing the student enough flexibility to adapt to the lower-quality CT input.

Key Results

The PCRD framework significantly outperformed standard training and other common distillation techniques:

  • 2D Performance: Improved mean Average Precision (mAP) by 1.66 percentage points over direct training on micro-CT data alone.
  • 3D Fidelity: When validated against synchrotron data, the PCRD reconstruction matched 89.97% of the reference fiber centerline length. For comparison, simple interpolation of the low-res volume only reached 75.08%.
  • Structural Accuracy: The model closely matched the reference for fiber volume fraction, diameter, and local spacing—metrics that are critical for downstream mechanical simulations.

Limitations and Future Work

While the results are impressive, the authors note a few hurdles:

  • Anisotropy: The method primarily improves resolution in the 2D transverse plane. The resolution along the fiber direction (the Z-axis) is still largely limited by the original micro-CT sampling.
  • Structural Complexity: This study focused on densely packed, relatively uniform fibers. The effectiveness of ACSD on highly heterogeneous or sparse microstructures (where "informative relationships" might be harder to find) remains to be seen.
  • Future Directions: The authors suggest extending the method to reconstruct along multiple directions to achieve isotropically high resolution.

Summary updated

Abstract

High-resolution three-dimensional (3D) microstructure characterization remains difficult to undertake at scale because volumetric imaging at the required spatial resolution is expensive, time-consuming or destructive. In many settings, however, fine-resolution two-dimensional (2D) imaging and lower-resolution volumetric measurements are both readily available. We introduce a Privileged Cross-modal Relational Distillation (PCRD) framework that leverages fine-resolution 2D observations during training to improve specimen-specific 3D reconstruction from lower-resolution volumetric imaging alone. We demonstrate PCRD on pultruded carbon fiber-reinforced polymer composites, using scanning electron microscopy as the privileged modality and micro-CT as the inference input. The framework transfers spatial organization by matching similarity distributions defined around corresponding spatial anchors in the teacher and student representations, avoiding direct alignment of heterogeneous feature vectors. In 2D cross-sectional reconstruction, PCRD improves mAP by 1.66 percentage points over direct micro-CT training. Independent 3D validation against high-resolution synchrotron X-ray CT data shows close agreement in fiber volume fraction, diameter and local spacing. The PCRD reconstruction matches 89.97% of the reference fiber centerline length, compared with 75.08% for the interpolated low-resolution volume.
BibTeX
@misc{sui2026learning,
  author = {Wenbo Sui and Lars P. Mikkelsen and Jens W. Andreasen and Daniel Lichau and Harold Phelippeau and Philip J. Withers},
  title = {{Learning from high-resolution 2D images to reconstruct 3D microstructures}},
  year = {2026},
  eprint = {2611.02204},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.02204},
}
Figure