Programs that generate graphs, such as molecule generators, are slow because the best of them are iterative. A diffusion model, which is a generator that learns to turn random noise into a sample by removing a little noise at a time, needs tens to hundreds of sequential network calls per graph, and those calls cannot run in parallel. Fast one-step generators exist, but they assume the data lives in a continuous space such as pixels or 3D points. Graphs are discrete: an atom is either oxygen or carbon, and a bond is single or double. We propose to remove that mismatch with an idea from graph foundation models. Describe every node and edge label in plain language ("oxygen atom, charge −0.5", "aromatic bond"), embed each description once with a frozen language model, and treat the resulting vectors as a fixed codebook. The generator then works on continuous vectors where one-step methods apply. At the end, each generated vector is snapped to the nearest codebook entry, which recovers a discrete, valid label.
The central claim can be tested. Language embeddings arrange labels so that similar meanings sit close together, and we expect this to make the noise-to-data paths simpler and less curved than paths through arbitrary or one-hot label encodings. Less curved paths are what allow a single step to land accurately. If the claim holds, natural-language feature unification does more than let one model read many graph domains. It also becomes the ingredient that makes one-step generation of text-attributed graphs work, and an ablation that swaps in random codebooks decides the question directly.
What exists and what is missing
Two lines of work meet here without overlapping. On the generation side, DeFoG applies discrete flow matching, a method that moves graphs between label states through a learned continuous-time Markov chain, and adds time distortion and target guidance at sampling. It reaches 92.8% validity on MOSES in 50 steps, against 500–1000 for earlier diffusion models. CatFlow recasts flow matching as predicting categorical end-points with a cross-entropy loss and reports 99.81% validity on QM9. Neither model has any notion of text, and both remain iterative.
MeanFlow supplies the one-step method we reuse. It trains a network to predict the average velocity over an interval rather than the instantaneous velocity, which gives a 1-step FID of 3.43 on ImageNet 256×256 with no teacher model. 3D MeanFlow shows the method carries over to unordered point sets, at about 5 ms per shape. Both papers work only on data that is naturally continuous. 3D MeanFlow also reports that diversity drops when path curvature is not controlled, which is exactly the property our codebook geometry is supposed to improve. That paper therefore both motivates and constrains our design.
On the text side, UTGDiff is the closest existing work. It puts atoms and instruction words into a single token vocabulary inside a RoBERTa-initialized Transformer and generates molecules by masked discrete diffusion over a full step schedule. It reports an FTS of 0.724 and 37.4% exact match on ChEBI-20. We keep its text–graph sharing and replace both its discrete masking process and its multi-step schedule.
UDPD runs continuous diffusion on text-attributed graphs, but only to clean up node representations for classification. It never generates graphs. The foundation-model papers (OFA, UniGraph) motivate the shared text space and are purely discriminative. They also warn about a "textualization tax", the cost of running large language models over every node. Our design avoids that tax because the encoder runs once per label type, not once per node. A molecule vocabulary has only dozens of entries.
Core Claim
Represent every node and edge label as a fixed, low-dimensional projection of its language-model embedding. Train a permutation-equivariant graph Transformer with the MeanFlow objective on these continuous node and edge tensors, conditioned on an instruction text, and decode by nearest-codebook snapping. We claim this model will match multi-step discrete graph generators at 1–2 network calls. We also claim the gain depends on the semantic geometry of the codebook and not merely on making the representation continuous.
Research Program
Each label $k$ (atom type with charge, bond type, or "no edge") gets a description $d_k$. We encode it with a frozen sentence encoder and reduce it with PCA to a vector $c_k \in \mathbb{R}^m$, where $m \approx 16$ keeps the $N \times N$ edge tensor affordable. A clean graph $x$ is the stack of these vectors for all nodes and node pairs. With noise $\epsilon$ and times $0 \le r < t \le 1$, the network $u_\theta(z_t, r, t \mid s)$, conditioned on instruction $s$, is trained so that its prediction satisfies the MeanFlow identity $u = v - (t-r)\,\tfrac{d}{dt}u$. Here $v = \epsilon - x$ is the instantaneous velocity along the straight noise-to-data path, and $u$ is the average velocity over $[r,t]$. The time derivative is computed with a Jacobian-vector product, which MeanFlow reports adds about 16% training overhead.
One-step sampling computes $\hat{x} = \epsilon - u_\theta(\epsilon, 0, 1 \mid s)$ and then snaps each entry:
$$\hat{k}_i = \arg\min_k \|\hat{x}_i - c_k\|_2 ,$$
where $\hat{x}_i$ is the generated vector at node or pair $i$. We add a small auxiliary loss that classifies $\hat{x}$ against the codebook entries, in the spirit of CatFlow's end-point prediction. Its job is to keep snapping errors rare.
The backbone follows UTGDiff: RoBERTa initialization, with instruction tokens and graph tokens in one sequence and connectivity entering as an attention bias. Holding the backbone fixed means differences in results come from the generative process and not from capacity.
There are two analyses. The first measures path curvature, as the gap between one-step and 50-step reconstructions, for each codebook type. The second tests cross-domain sharing. We pool ZINC250k with ChEBI-20 under one textual vocabulary and check whether co-training helps or hurts. The text-space GFM benchmark found positive transfer for graph-level tasks and negative transfer for node-level ones, so it is an open question which side whole-graph generation falls on.
Baselines
The naive reference point is the same backbone and the same codebook, trained with ordinary flow matching and sampled with 1, 2, 4 and 50 uniform Euler steps. This shows how much of any result comes from the average-velocity objective rather than from the representation.
The strongest text-conditioned competitor is UTGDiff run at its native schedule, for which ChEBI-20 FTS 0.724 and exact match 37.4% are reported. Lacuna has no reported latency or step count for it, so we will re-time it on our hardware. We will also run it truncated to 1–4 steps, because the fair question is quality at a fixed wall-clock budget. We use the same data split, the same RoBERTa initialization, and the same number of training GPU-hours for both models, and each gets an equal hyperparameter search of about 20 trials.
The strongest unconditional fast competitor is DeFoG with time distortion and target guidance, at 92.8% MOSES validity in 50 steps. We will also evaluate it at 1, 2 and 4 steps. We beat it if we match its validity, uniqueness and structural MMD at 50 steps while using at most 2 network calls, or if we are clearly better at matched calls.
MolT5-large, at FTS 0.684 on ChEBI-20, stands in for autoregressive text-to-SMILES generation. We compare it on quality and on measured latency.
The key ablation keeps everything fixed and replaces the language codebook first with random Gaussian vectors of the same dimension and norm, and then with one-hot encodings projected to $m$ dimensions. If one-step quality does not degrade under these swaps, our claim about semantic geometry is false, and the contribution shrinks to "continuous relaxation plus MeanFlow".
Why not just…?
Why not just run DeFoG or UTGDiff with fewer steps? DeFoG's sampling-time tricks are exactly the kind of training-free acceleration that could be good enough. The file does not report its quality below 50 steps, and MeanFlow shows that one-step results from methods without the average-velocity objective are much worse: iCT-XL reaches FID 34.24 against MeanFlow's 3.43 on ImageNet. That evidence is from images, not graphs, so we treat this alternative as a real threat. The step sweeps in our baselines settle it directly. If DeFoG at 4 steps matches us, our case rests only on the 1-step regime.
Why not just distill UTGDiff into a few-step student? Progressive distillation is the standard way to compress a trajectory, but it needs a trained teacher and several rounds of training, and the student inherits the teacher's discrete masking process. We will distill UTGDiff to 2 and 4 steps under the same total compute as our model, teacher training included. A distilled student that matches us at equal compute would weaken the claim. Distillation does not explain any gap we find between random and language codebooks.
Why not just generate SMILES strings with an autoregressive language model? Such a model makes one pass, but it emits tokens one at a time, so its latency grows with molecule length. On ChEBI-20 it trails UTGDiff in quality (MolT5-large FTS 0.684 against 0.724). We report both quality and wall-clock time to show where each method sits on the trade-off between them.
Why not just use one-hot labels with CatFlow-style simplex flows? This is precisely the alternative our ablation tests. If one-hot encodings work as well as language embeddings at one step, the text grounding adds nothing to speed, and we will say so.
Proposed Experiments
On ChEBI-20 we report FTS, exact match and validity at 1, 2, 4 and 50 network calls, together with milliseconds per molecule. On QM9, MOSES and ZINC250k, unconditionally, we report validity, uniqueness and novelty, and on synthetic Planar and SBM graphs we report structural MMD, again at each call budget. Planar and SBM nodes have no attributes, so their nodes carry the single description "node" and edges carry "edge" or "no edge". This tests the topology half of the method alone.
Three checks decide the outcome. First, one-step ChEBI-20 FTS should come within 0.02 of UTGDiff's 0.724 while running at least an order of magnitude faster. Second, the random-codebook ablation should show a clear one-step gap that shrinks as the number of steps grows, which is the signature we would expect if semantic geometry reduces curvature. Third, the snapping error rate should stay below 1% of entries. If the second check fails, we will report the method as a fast continuous relaxation and not as evidence that a language space matters for sampling speed.
References
- DeFoG: Discrete Flow Matching for Graph Generation
- Variational Flow Matching for Graph Generation
- Mean Flows for One-step Generative Modeling
- 3D MeanFlow: One-Step Point Cloud Completion and Generation via Average-Velocity Transport
- Instruction-Based Molecular Graph Generation with Unified Text-Graph Diffusion Model
- Bridging Structure and Semantics: Uncertainty-Modulated Dual-Path Diffusion for Robust Text-Attributed Graph Learning
- One For All: Towards Training One Graph Model For All Classification Tasks
- UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs
- Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights