TL;DR
This paper introduces a method to adapt crystal generative models at inference time using Diffusion Tree Sampling (DTS), a search procedure that treats the denoising process as a tree of possibilities. By incorporating importance-corrected conditional proposals and multi-objective optimization, the authors enable these models to discover stable crystals with specific, often conflicting, physical properties without needing to retrain the underlying "foundation" models.
What Problem Does This Paper Address?
Generative diffusion models like MatterGen have become powerful tools for proposing new crystal structures. However, simply sampling from these models is often insufficient for real-world materials discovery. Researchers usually need crystals that satisfy multiple constraints: they must be thermodynamically stable, belong to specific symmetry groups, and possess desirable properties like a high bandgap or bulk modulus.
Existing solutions typically involve fine-tuning a new model for every new objective—which is computationally expensive—or using differentiable guidance, which doesn't work for many non-differentiable physical properties (like crystal stability). This paper tackles the challenge of inference-time adaptation: steering a fixed, pre-trained model toward complex goals without modifying its weights.
Methods
The core technique is Diffusion Tree Sampling (DTS), which applies Monte Carlo Tree Search (MCTS) to the denoising trajectory of a diffusion model. Instead of a single path from noise to crystal, DTS explores a tree of partial trajectories, focusing compute on branches that show promise according to a reward function (e.g., predicted stability).
The authors contribute two key technical enhancements:
- Importance-Corrected Guided Proposals: When a specialized conditional model (e.g., one fine-tuned for a specific property) is available, it can be used to suggest better denoising steps. However, naively using it "double-tilts" the distribution. The authors use importance sampling to correct this bias, allowing the search to benefit from the conditional model's local guidance while maintaining the integrity of the target distribution.
- Multi-Objective DTS (MO-DTS): Real-world design involves trade-offs (e.g., high dielectric constant vs. high bandgap). MO-DTS maintains separate value estimates for each objective and samples a "preference" vector for each traversal. This allows a single search tree to efficiently map out the entire Pareto front—the set of optimal trade-offs—sharing computation across different goals.
Datasets
The authors evaluate their method using two state-of-the-art crystal diffusion models:
- MatterGen: Trained on a massive dataset of known crystals, used here for stability search, rare space group generation, and multi-objective property optimization.
- DiffCSP: A faster model used to demonstrate how DTS scales with increased inference-time compute.
- MP-20: A standard subset of the Materials Project database used for training baselines and evaluating novelty.
Key Insights and Ideas
- Trajectory Reuse is a Superpower: By branching the denoising process, DTS reuses the "expensive" early steps of generation. This allows it to produce significantly more unique, stable crystals for the same total computational budget (NFEs) compared to standard sampling.
- The "Double-Tilt" Trap: A major insight is that using a conditional model as a sampler and applying a reward tilt results in a distorted distribution. Importance sampling is the mathematical "glue" that lets you use specialized models as efficient search proposals without losing the desired target distribution.
- Optimistic Selection: In the multi-objective setting, the authors show that their scalarization approach is "optimistic"—it prioritizes nodes where objectives trade off, ensuring the search effectively explores the edges of the Pareto front.
Key Results
- Improved Stability: On the MatterGen model, DTS increased the absolute count of "Stable, Unique, and Novel" (SUN) structures by over 60% compared to standard best-of-N sampling at the same compute budget.
- Rare Space Groups: For extremely rare crystal symmetries (occurring in <0.2% of training data), DTS produced an order of magnitude more on-target structures than baselines.
- Pareto Front Discovery: In the conflicting task of maximizing both dielectric constant and bandgap, MO-DTS successfully generated a diverse set of candidates along the Pareto front, outperforming baselines that used multiple separate conditional models.
- Scaling: The benefits of DTS grow with compute. As shown in Figure 2 (not pictured here but discussed in text), the stability rate continues to climb with more search steps, whereas standard sampling hits a plateau.
Limitations and Future Work
- Reward Fidelity: The search is only as good as the reward function. The authors use learned surrogate models (like ALIGNN) for properties; if these surrogates have systematic errors, the search will amplify them. Validation with high-fidelity Density Functional Theory (DFT) remains essential.
- Backbone Consistency: Importance sampling works best when the proposal and base models share a similar "backbone." Using wildly different models as proposals might lead to high variance in weights.
- Beyond Crystals: While demonstrated on materials, the authors suggest this framework is generalizable to other "AI for Science" domains, such as protein design or small-molecule discovery.
Summary updated