Every field that builds generative models eventually needs a ruler: a way to say, with a number, whether one model's outputs are better than another's. In de novo molecular design, the accepted ruler is the Fréchet ChemNet Distance (FCD), which passes real and generated molecules through a shared neural network and compares the two resulting clouds of embedding vectors as if each were a Gaussian Fréchet ChemNet Distance: A Metric for Generative Models for Molecules in Drug Discovery. Two problems with this ruler are now documented. First, the Gaussian assumption is false: real chemical space is clustered and multi-modal, and a speaker-verification study has already shown that applying Gaussian statistics to non-Gaussian neural embeddings silently degrades the resulting metric Neural Discriminant Analysis for Deep Speaker Embedding. Second, the ruler is size-sensitive: FCD values shift systematically with the number of generated molecules and only converge above roughly 10,000 designs, which means small evaluation budgets can invert model rankings How evaluation choices distort the outcome of generative drug discovery — a direct contradiction of the original claim that 5,000 molecules suffice "with negligible variance" Fréchet ChemNet Distance: A Metric for Generative Models for Molecules in Drug Discovery.
The proposal: stop using normalizing flows as one more generative architecture to be graded, and start using them as the grading instrument itself. A normalizing flow — an invertible neural map with an exactly computable change-of-variables density — can be trained once on the embedding vectors of a reference chemical library, so that the reference distribution becomes exactly a standard Gaussian in the transformed space. In that transformed space the Fréchet formula is no longer an approximation on the reference side, and, more importantly, the same flow assigns an exact, model-independent log-density to every individual generated molecule, giving an evaluation score that needs no batch statistics at all.
Core Claim
Training a small normalizing flow on ChemNet embeddings of a fixed reference set, and then computing distributional distance in the flow-transformed space (FlowFCD) plus per-molecule reference log-likelihoods, yields a molecular generative-model metric that is (i) more rank-stable across library sizes than standard FCD, (ii) applicable to any generative architecture — string models, graph variational autoencoders, diffusion models, even genetic algorithms — because it never consults the generator's internals, and (iii) better correlated with downstream physical checks (docking scores, medicinal-chemistry filters) than either standard FCD or the generative model's own likelihoods.
Background and Positioning
The literature at this intersection treats flows exclusively as the thing being measured. GraphNVP, MolGrow, MolHF and their relatives build invertible networks over molecular graphs and are then scored by the usual validity/uniqueness/FCD suite GraphNVP: An Invertible Flow Model for Generating Molecular Graphs, MolGrow: A Graph Normalizing Flow for Hierarchical Molecular Generation, MolHF: A Hierarchical Normalizing Flow for Molecular Graph Generation. None of them, nor any benchmark paper we found on Lacuna, uses a flow's exact-density property as an evaluation tool for other generators. Two papers motivate the design from opposite sides. Neural Discriminant Analysis for Deep Speaker Embedding supplies the mechanism: it showed, via skewness and kurtosis measurements, that a maximum-likelihood flow removes non-Gaussianity from deep embeddings and thereby rescues a downstream Gaussian estimator (PLDA) in speaker verification — but it never left acoustics and never touched distributional distances. How evaluation choices distort the outcome of generative drug discovery supplies the need: its authors binned generated molecules by their own model's sequence likelihood and found an exploration–exploitation trade-off, but they explicitly flag as open the problem that these internal likelihoods are model-specific and cannot compare architectures. This proposal closes that loop: the flow is the external, architecture-neutral likelihood assessor they asked for.
Research Program
The built artifact is a reference density model in embedding space, not a molecule generator. Concretely, we take the penultimate-layer ChemNet activations used by FCD Fréchet ChemNet Distance: A Metric for Generative Models for Molecules in Drug Discovery as our embedding, train a conditional-free normalizing flow $g$ — built from neural spline layers, whose invertible monotonic couplings give expressive non-linear maps with linear-time Jacobian determinants, as used for gravitational-wave posterior estimation in Statistically-informed deep learning for gravitational wave parameter estimation — by maximum likelihood on embeddings of the reference set $x_{ref}$ (a curated ChEMBL split, following the GuacaMol protocol GuacaMol: Benchmarking Models for de Novo Molecular Design). Training follows the NDA recipe Neural Discriminant Analysis for Deep Speaker Embedding, and we verify Gaussianization of the reference by the same skewness/kurtosis diagnostics that paper used.
The metric has two layers. For a generated set $x_{gen}$ with embeddings $h_{gen}$, FlowFCD is the ordinary Fréchet distance computed between the standard normal $\mathcal{N}(0, I)$ (the transformed reference, exact by construction) and a Gaussian fitted to $g(h_{gen})$:
$$\text{FlowFCD}^2 = \|\mu_g\|_2^2 + \mathrm{Tr}\left(\Sigma_g + I - 2\Sigma_g^{1/2}\right)$$
where $\mu_g, \Sigma_g$ are the empirical mean and covariance of the transformed generated embeddings. Independently, each molecule $x$ receives a pointwise score by the flow's exact change of variables, $s(x) = \log p_z(g(h(x))) - \log \left|\det \frac{\partial g(h(x))}{\partial h(x)}\right|$, the log-density the reference library assigns to that molecule's embedding. Averaging $s$ over any number of generated molecules gives a sample-size-invariant mean score; binning molecules by $s$ reproduces the Özçelik likelihood-bin analysis How evaluation choices distort the outcome of generative drug discovery with an external, cross-model score. We will be honest about a limitation this design does not erase: FlowFCD still fits a Gaussian to the generated set in transformed space, so if a generator's embedding distribution is so multi-modal that even the flow cannot Gaussianize it, residual approximation remains. The pointwise score $s(x)$ carries no such assumption and is our primary hedge; the experiments below are designed to detect exactly this failure mode rather than assume it away.
Proposed Experiments
The first experiment measures rank stability. We generate libraries from a deliberately heterogeneous model pool spanning architectures: a SMILES LSTM (the GuacaMol top performer, which achieved 77% compound quality, matching the "Best of Dataset" baseline GuacaMol: Benchmarking Models for de Novo Molecular Design), a character VAE (whose validity on the 250k ZINC split collapsed to less than 1% A Systematic Assessment of Deep Learning Models for Molecule Generation), MolGAN (20% unique molecules on QM9, a documented mode-collapse case A Systematic Assessment of Deep Learning Models for Molecule Generation), graph models of the GraphINVENT/DiGress family From In Silico to In Vitro: Evaluating Molecule Generative Models for Hit Generation, and a simple genetic algorithm A Fresh Look at De Novo Molecular Design Benchmarks. For library sizes from 100 to 50,000 we bootstrap sublibraries and record how often the metric's ranking of models changes. The target is quantified: standard FCD is known not to converge below roughly 10,000 designs How evaluation choices distort the outcome of generative drug discovery; FlowFCD and the mean pointwise score should show materially lower rank-flip rates at sizes 500–5,000. The second experiment measures external validity: on the target-specific ChEMBL splits and ExCAPE-DB fine-tuning tasks (DRD3, PIN1, VDR) used by Özçelik and Grisoni How evaluation choices distort the outcome of generative drug discovery, and on the SMINA docking benchmark with its eight targets such as 5-HT1B and D2 Generative Models Should at Least Be Able to Design Molecules That Dock Well: A New Benchmark, we test whether per-molecule flow scores predict docking score and hit-like filter compliance (the 150–350 Da medicinal-chemistry filters under which generally-trained models scored below 12% compliance in From In Silico to In Vitro: Evaluating Molecule Generative Models for Hit Generation) better than internal CLM likelihoods do, using the same binning protocol. The third experiment is the decisive head-to-head described under Baselines: standard FCD versus FlowFCD on identical libraries.
Baselines
The naive reference point is the "do nothing" library: random molecules drawn from ZINC, or the best molecules selected directly from the training set. This matters because A Fresh Look at De Novo Molecular Design Benchmarks showed that trivial baselines like Dataset-Best saturate scalar metrics, and Generative Models Should at Least Be Able to Design Molecules That Dock Well: A New Benchmark showed CVAE and GVAE failed to outperform the mean docking score of random ZINC molecules. A distributional metric that scores a trained generative model as equivalent to a random ZINC draw of the same size is not measuring anything; both FlowFCD and standard FCD must separate these libraries decisively, and this separation is a sanity gate before any other comparison.
The strongest competitor is standard FCD with empirical mean and covariance, exactly as defined by Preuer et al. Fréchet ChemNet Distance: A Metric for Generative Models for Molecules in Drug Discovery and used throughout GuacaMol GuacaMol: Benchmarking Models for de Novo Molecular Design. It is the community standard and the incumbent we must beat on its own turf. The comparison is run fairly: both metrics consume the identical ChemNet embeddings of the identical generated libraries, at matched library sizes, and the flow's one-off training cost is reported separately from per-evaluation cost (FCD evaluation is itself cheap, so the honest claim we aim for is better statistical behavior at matched or slightly higher compute, not speed). Beating it means two concrete results: lower model-ranking variance under bootstrap resampling at small library sizes (where the incumbent is documented to be unstable, converging only above 10,000 designs How evaluation choices distort the outcome of generative drug discovery), and higher rank correlation with the downstream docking and hit-filter outcomes. A third baseline is the internal-likelihood analysis of Özçelik and Grisoni How evaluation choices distort the outcome of generative drug discovery, restricted to the SMILES models it can score; the flow's pointwise score must match or beat its likelihood bins as a predictor of bioactivity-related outcomes while additionally covering models it cannot score at all.
The key ablation isolates the Gaussianizing ingredient: we replace the trained flow with the identity map, which reduces FlowFCD exactly to standard FCD, and we also test a flow trained on the generated set rather than the reference (or on shuffled labels), which should destroy the metric's behavior if the reference-likelihood semantics are doing the work rather than the extra parameters. No reported result found on Lacuna for any flow-based correction to FCD, so the ablation, not a literature number, defines the expected delta.
Why not just…?
Why not just generate 10,000+ molecules and use standard FCD? This is the most serious alternative, and for large-library distribution-learning benchmarks it may genuinely suffice — the honest position is that FlowFCD's advantage there is statistical exactness, not necessity. But the size trap has teeth exactly where evaluation is expensive: A Fresh Look at De Novo Molecular Design Benchmarks showed model rankings invert across oracle budgets, and docking-based evaluation Generative Models Should at Least Be Able to Design Molecules That Dock Well: A New Benchmark restricts libraries to 250 molecules per target, far below FCD's documented convergence threshold How evaluation choices distort the outcome of generative drug discovery. Our bootstrap experiment settles this directly: if FlowFCD's rank stability at matched sizes equals standard FCD's, the proposal's value collapses to the small-sample and pointwise-score regimes, and we will report that plainly.
Why not just use heuristic drug-likeness filters? Because they are known to be gameable and saturating: Tripp et al. showed simple baselines achieve near-perfect QED A Fresh Look at De Novo Molecular Design Benchmarks, GuacaMol's authors documented reinforcement-learning agents exploiting scalar rewards into implausible structures GuacaMol: Benchmarking Models for de Novo Molecular Design, and the hit-like filters reject generally-trained output below 12% compliance with no gradient information about near-misses From In Silico to In Vitro: Evaluating Molecule Generative Models for Hit Generation. The flow score is continuous and reference-relative rather than binary; the correlation experiment tests whether that continuity actually tracks downstream filter compliance.
Why not just use each model's own likelihood? For autoregressive string models this works and is exactly what Özçelik and Grisoni did — but it cannot score graph VAEs, diffusion models, or genetic algorithms, and its own authors found high internal likelihood favors repetitive known-chemistry motifs over novel actives, and flagged model-independent likelihood reporting as an open problem How evaluation choices distort the outcome of generative drug discovery. The flow likelihood is the same analysis made cross-architecture; the binning experiment compares the two head-to-head on the models where both exist, which is the fair test.
Why not just dock everything with SMINA? Cost and coverage: docking oracles are too expensive for the $10^4$–$10^5$ evaluation budgets where rankings are decided A Fresh Look at De Novo Molecular Design Benchmarks, and docking measures single-target affinity, not distributional coverage of chemical space. Docking is therefore our ground-truth validator, not a competitor: the docking scores from Generative Models Should at Least Be Able to Design Molecules That Dock Well: A New Benchmark are what FlowFCD must correlate with to be trusted.
Decisive Checks
The proposal stands or falls on three numbers. First, rank-flip rate: the fraction of bootstrap resamples at library sizes 500–5,000 in which FlowFCD orders two models differently from the full-library ordering, compared against standard FCD's rate — the incumbent's instability at these sizes is documented at convergence "when more than 10,000 designs are considered" How evaluation choices distort the outcome of generative drug discovery. Second, the rank correlation between flow pointwise scores and SMINA docking scores on the eight-target benchmark, against the same correlation for internal CLM likelihoods. Third, the identity-map ablation: if replacing the flow with the identity changes nothing, the Gaussianization story is wrong and the honest conclusion is that ChemNet embedding non-Gaussianity does not matter for FCD-scale comparisons — itself a result the field currently assumes without testing.
References
- Fréchet ChemNet Distance: A Metric for Generative Models for Molecules in Drug Discovery
- Neural Discriminant Analysis for Deep Speaker Embedding
- How evaluation choices distort the outcome of generative drug discovery
- GraphNVP: An Invertible Flow Model for Generating Molecular Graphs
- MolGrow: A Graph Normalizing Flow for Hierarchical Molecular Generation
- MolHF: A Hierarchical Normalizing Flow for Molecular Graph Generation
- Statistically-informed deep learning for gravitational wave parameter estimation
- GuacaMol: Benchmarking Models for de Novo Molecular Design
- A Systematic Assessment of Deep Learning Models for Molecule Generation
- From In Silico to In Vitro: Evaluating Molecule Generative Models for Hit Generation
- A Fresh Look at De Novo Molecular Design Benchmarks
- Generative Models Should at Least Be Able to Design Molecules That Dock Well: A New Benchmark