In natural language processing, we tokenize text into sub-words because characters are too granular and full words are too sparse. In chemistry, a similar tension exists. For years, researchers represented molecules as SMILES strings—sequences of characters like C, (, and =. However, as Taicheng Guo and colleagues established in What indeed can GPT models do in chemistry?, standard Large Language Models (LLMs) often fail to "understand" these strings because their tokenizers fragment them into meaningless pieces that ignore atomic boundaries and functional groups.
The emerging consensus across the field is a shift toward fragment-based tokenization: treating molecules not as strings of characters, but as sequences of chemically meaningful "motifs" or building blocks.
The Semantic Gap in Chemical Strings
The fundamental problem with character-level representations is a lack of chemical semantics. As Feiyang Cai and Gang Li point out in MolLangBench, standard Byte-Pair Encoding (BPE) might merge a carbon atom with a nearby parenthesis, or split a benzene ring in half. This makes it nearly impossible for a model to perform basic tasks like counting atoms or identifying reactive sites.
By contrast, motif-level tokenization—advocated by Chang Liao and Yu Mei in their survey of LLMs in chemistry—segments molecular structures into functional groups like hydroxyls or carboxyls. This ensures the model's internal vocabulary aligns with the conceptual framework of human chemists. When a model sees a "token" for a specific ring system, it doesn't have to reconstruct the concept of "aromaticity" from raw characters; the property is baked into the token itself.
Building with Bricks instead of Atoms
Why does this shift improve generative performance? It comes down to the search space and structural integrity.
- Reduced Sequence Length: Representing a molecule as a collection of fragments (like the FU-SMILES approach in FragGPT) significantly shortens the sequence length. Jie Yue argues this mitigates "exposure bias"—the tendency for errors to compound during long autoregressive generation.
- Hardcoded Validity: In Group SELFIES, the authors demonstrate that using tokens for entire functional groups provides an inductive bias. It is much harder for a model to generate a chemically "impossible" structure if its primary building blocks are already validated fragments.
- Geometric Integrity: For 3D drug design, Cong Fu shows in Frag2Seq that tokenizing by geometric fragments preserves local spatial relationships that are often lost when generating atom-by-atom.
Bridging the Gap to the Laboratory
Perhaps the most practical advantage of this direction is synthetic accessibility. A common critique of AI-designed molecules is that they are "monsters"—structures that look good on paper but are impossible to synthesize.
Carl Edwards and the team behind mCLM addressed this by "front-loading" the constraints of laboratory robots into the model's vocabulary. By using fragments derived from real-world chemical reactions (often using the BRICS decomposition algorithm), the model generates molecules that are 98% "makeable." Similarly, Vineet Thumuluri and Michael K. Gilson found that replacing simple decoders with fragment-based tokenization significantly improved the binding affinity and synthetic ease of generated drugs.
Explainability through Partitioning
Finally, fragment-based models are inherently more interpretable. In MoleX, Xu Sun demonstrates that mapping high-dimensional LLM embeddings onto discrete functional groups allows the model to "explain" its predictions. If a model predicts a drug will be toxic, it can point to the specific fragment (token) responsible for that prediction.
This alignment between AI reasoning and medicinal chemistry is further explored in Lamole by Zhenzhong Wang, where functional group tokenization allows for "molecular editing"—human experts can interact with the model's suggestions by swapping out specific building blocks, much like a chemist would at a whiteboard.
By moving from the "alphabet" of atoms to the "vocabulary" of fragments, researchers like Dong Xu have shown that molecular language models can achieve lower irreducible loss and better scaling behavior, ultimately making them more efficient and reliable partners in scientific discovery.