Nahual: A Sequence Model for Language and Atoms

Austin H. Cheng, Marcel Müller, Andreas Burger, Yeonghun Kang, Tsz Wai Ko, Marta Skreta, Luka Mucko, Ella Miray Rajaonson, Cher Tian Ser, Jérôme F. Gonthier, Alex Zook, Varinia Bernales, Alán Aspuru-GuzikarXiv 2026 · arXiv

TL;DR

Nahual is a generalist, decoder-only transformer that bridges the gap between natural language and 3D molecular geometry. By introducing a novel "next-token diffusion" mechanism, the researchers created a single model capable of performing 12 diverse chemistry tasks—from predicting crystal structures to finding transition states—often outperforming specialized, task-specific models.

What Problem Does This Paper Address?

In the current AI landscape, we have a "modality gap" in chemistry. On one side, Large Language Models (LLMs) are excellent at reasoning through text but struggle with 3D coordinates (serializing numbers into text digits is inefficient and loses spatial continuity). On the other side, specialized models (like AlphaFold) are masters of 3D geometry but lack the flexible, conversational controllability of language.

To solve complex problems—like "find me a transition state for this reaction" or "solvate this drug molecule with ten waters"—researchers currently have to glue these systems together using "agentic" frameworks that call specialized tools. This paper argues that this gap isn't fundamental. Just as NLP moved from bespoke architectures for translation and tagging to monolithic LLMs, chemistry is ready for a unified model that treats atoms and words as part of the same sequence.

Methods

The authors propose Nahual (named after a Mesoamerican shapeshifting sorcerer), a decoder-only autoregressive model. Its core innovation is next-token diffusion:

  1. Unified Tokenization: Every token in a sequence is a pair $(a, x)$, where $a$ is a discrete value (a word ID or an atomic number) and $x$ is a continuous 3D coordinate (zero-padded for words).
  2. Two-Stack Architecture:
    • Stack 1 (Discrete): A Mixture-of-Transformers (MoT) initialized from Qwen3-0.6B. It processes the sequence to predict the next discrete token (e.g., "What is the next atom type?").
  3. Stack 2 (Continuous): A causal diffusion transformer that acts as a denoiser. It predicts the 3D position of the next atom.
  4. Next-Token Diffusion: Unlike standard diffusion which might noise an entire image or sequence at once, Nahual denoises the current token while attending to the clean (already generated) tokens before it. This allows the model to handle variable-length molecules without the context-window explosion seen in text-only LLMs.

Datasets

Nahual was trained on a massive, weighted mixture of 12 task families (~1.4 million sequences):

  • Structure Search: Transition1x (transition states), NCI Atlas (noncovalent interactions), LnQM (lanthanide complexes), and MP-20 (crystals).
  • Adsorption: Alloy, FG, and OCD datasets for molecules on metal surfaces; ODAC25 for metal-organic frameworks (MOFs).
  • Properties & Generation: QM9 (small molecules), GEOM-DRUGS (conformer search), and OC20NEB (reaction trajectories).
  • New Benchmark: The authors introduced QCG-10, a new microsolvation dataset featuring drug-like molecules solvated with 1–10 explicit water molecules.

Key Insights and Ideas

  • Inductive Bias vs. Scale: The authors argue that LLMs fail at 3D geometry not because they lack capacity, but because they have the wrong inductive bias. By treating 3D coordinates as continuous values rather than strings of digits, Nahual maintains spatial continuity and reduces context length by roughly 4x.
  • The "Generalist" Advantage: Training on 12 diverse tasks simultaneously allows for "positive transfer." For example, the model's understanding of small molecule geometry helps it predict complex crystal structures.
  • Self-Correction (Self-Reward): Because Nahual is both a generator and a property predictor, it can perform "inference-time scaling." It can generate five candidate molecules, predict their properties itself, and return the one that best matches the user's prompt.

Key Results

Nahual demonstrates that a single set of weights can beat specialized SOTA models:

  • Crystal Prediction: On the MP-20 benchmark, it achieved a 79.9% match rate, surpassing the specialized DiffCSP model (77.9%).
  • Property-Conditional Generation: On the QM9 dataset, Nahual reached state-of-the-art performance across all properties when using its "self-reward" sampling method.
  • Metal Complexes: It achieved a 73.3% connectivity match rate, dwarfing the 10.2% achieved by Architector, a standard automated tool for building metal complexes.
  • Transition States: It reached a median RMSD of 0.197 Å, outperforming the specialized TSDiff model.

Limitations and Future Work

  • Catastrophic Forgetting: Because it was trained on fixed templates for chemistry tasks, the model "forgot" how to speak natural language fluently. Future versions will need to mix in general instruction-following data.
  • Numerical Precision: Predicting properties as text digits (via cross-entropy) is less accurate than using a continuous regression head.
  • Inference Speed: Because each atom requires a multi-step diffusion process through the second stack, Nahual is significantly slower than specialized "one-shot" models.
  • Scaling to Large Systems: Performance begins to degrade on systems larger than 350 atoms, particularly regarding "steric clashes" (atoms being placed too close together).

Summary updated

Abstract

Generative models play a rapidly growing role in the design and simulation of molecules. The most widely used applications of generative models are in large language models (LLMs), owing to the natural flexibility of language for specifying and answering queries. However, generalist LLMs do not effectively handle large sets of 3D coordinates, whereas specialist generative models of 3D molecules cannot reach the controllability afforded by language. To bridge these capabilities, we propose Nahual, a decoder-only autoregressive diffusion model for any sequence of text and 3D molecules. To enable expressive autoregression on continuous coordinates, we propose next-token diffusion, whereby a causal transformer denoises each token while attending to the clean tokens before it, without increasing context size. We train Nahual on a curated set of twelve 3D chemistry tasks ranging from microsolvation to molecular conformer search to adsorption on metal surfaces, with associated evaluation metrics and baselines. The result is a single set of end-to-end-trained weights that simultaneously performs all tasks and surpasses state-of-the-art models specialized for crystal structure prediction and property-conditional molecule generation. Our results demonstrate the viability of scaling generalist decoder-only models for native multimodal understanding and generation across language and chemistry. Code is available at https://github.com/the-matter-lab/nahual.
BibTeX
@misc{cheng2026nahual,
  author = {Austin H. Cheng and Marcel Müller and Andreas Burger and Yeonghun Kang and Tsz Wai Ko and Marta Skreta and Luka Mucko and Ella Miray Rajaonson and Cher Tian Ser and Jérôme F. Gonthier and Alex Zook and Varinia Bernales and Alán Aspuru-Guzik},
  title = {{Nahual: A Sequence Model for Language and Atoms}},
  year = {2026},
  eprint = {2611.19072},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19072},
}
Figure