Abstract
Molecular fingerprints are essential tools in cheminformatics, yet their reliance on hash functions introduces "hash collisions"—where distinct chemical substructures map to the same bit. This study investigates whether these collisions degrade machine learning performance in property prediction and Bayesian Optimization (BO). We find that while "Exact" (collision-free) fingerprints offer statistically significant improvements in regression accuracy, these benefits surprisingly do not extend to BO performance. Our results suggest that standard hashed fingerprints are "good enough" for iterative molecular discovery.
Introduction
In cheminformatics, molecules are typically represented as fixed-length bit vectors known as fingerprints (e.g., ECFPs or Morgan fingerprints). These vectors encode the presence of specific chemical substructures. Due to the vast number of possible substructures, hash functions are used to "fold" these features into a manageable vector size, typically 1024 or 2048 bits.
The pigeonhole principle dictates that this folding inevitably causes hash collisions. Intuitively, collisions should be problematic as they force models to treat distinct structural motifs as identical, leading to an overestimation of molecular similarity. This paper quantifies this overestimation bias and asks whether it hinders our ability to predict molecular properties or optimize for top drug candidates.
Methods
We compared three fingerprinting strategies using Gaussian Processes (GPs) with a Tanimoto kernel:
- Compressed Fingerprints: Standard RDKit-style fingerprints using hash-based folding (sizes 512 to 4096).
- Exact Fingerprints: A collision-free version preserving every unique substructure. We developed a custom GP implementation to compute Tanimoto similarity directly from raw substructure counts.
- Sort&Slice: A method that selects the most frequent substructures from a reference dataset (ZINC250k) to populate the vector, avoiding hash functions.
Performance was evaluated using the DOCKSTRING regression benchmark, focusing on five protein targets: PARP1, F2, KIT, ESR2, and PGR.
Results
Property Prediction Accuracy
Exact fingerprints consistently outperformed compressed versions across all targets. For the KIT target, the $R^2$ score improved from 0.728 (compressed 4096) to 0.745 (exact). Tukey’s HSD tests confirmed these improvements were statistically significant. Regression performance scaled monotonically with fingerprint size, asymptotically approaching the "Exact" fingerprint's performance.
The Accuracy-Optimization Disconnect
The most striking finding is that superior predictive accuracy does not guarantee better optimization. In BO simulations over 1,000 iterations, Exact fingerprints did not consistently outperform compressed or Sort&Slice methods.
As shown in the figure above, while Exact fingerprints achieved the best Area Under the Curve (AUC) for the ESR2 target, they were matched or outperformed on the other four targets. This suggests that the "perceived" similarity caused by hash collisions does not critically disrupt the search for optimal molecules.
Discussion and Conclusion
While hash collisions technically degrade model accuracy, the Sort&Slice method often performed as well as exact fingerprints. This suggests that retaining the most informative features is more critical than retaining all features.
Several limitations exist: the Sort&Slice method's calibration on ZINC250k may involve data leakage, and the DOCKSTRING landscapes might be simple enough that all methods quickly find top candidates. Furthermore, this study focused exclusively on GPs.
Ultimately, this research provides a practical message for practitioners: standard 2048-bit fingerprints are likely sufficient for the iterative process of molecular discovery, as the theoretical benefits of collision-free representations do not necessarily translate to faster optimization in practice.
Summary updated