Can Decision Trees Teach Large Language Models? Distilling Verbalized Knowledge for Molecular Property Prediction

Khiem Le, Sreejata Dey, Marcos Martínez Galindo, Vanessa Lopez, Ting Hua, Nitesh V. Chawla, Hoang Thanh LamarXiv 2026 · arXiv

TL;DR

Large Language Models (LLMs) struggle with molecular property prediction because they process chemical strings globally rather than recognizing local substructures. This paper introduces TreeKD, a method that trains specialist decision trees on molecular functional groups, translates their predictive rules into natural language, and feeds them directly into LLM prompts. This approach, combined with a novel "rule-consistency" ensembling technique, significantly boosts LLM accuracy across diverse biochemical benchmarks.

What Problem Does This Paper Address?

Predicting molecular properties—specifically ADMET (Absorption, Distribution, Metabolism, Excretion, and Toxicity) characteristics—is a cornerstone of early-stage drug discovery. While traditional machine learning models (like decision trees) excel at specific prediction tasks by analyzing local chemical substructures, they are rigid specialists. LLMs offer a promising "generalist" alternative that can handle multiple tasks via a unified natural language interface.

However, LLMs face a critical bottleneck in chemistry: they read SMILES strings (textual representations of molecules) as a whole, struggling to identify local functional groups. Consequently, they fail to recognize activity cliffs—scenarios where a tiny structural modification completely flips a molecule's biological property.

Figure 1: An example illustrates that the LLM sees a molecule as a whole, while a decision tree operates on each dimension of functional groups.

Methods

To bridge this gap, the authors propose TreeKD (Tree Knowledge Distillation), which uses natural language as a bridge to transfer structured logic from tree-based models into LLMs. The pipeline consists of four main stages:

  1. Extracting Functional Groups (FGs): The framework uses a cheminformatics tool (SMARTChemist) to deterministically count occurrences of the 40K most common functional groups within an input molecule.
  2. Building Specialist Models: For each target molecular property, a specialist decision tree is trained directly on these functional group counts.
  3. Distilling Knowledge via Verbalization: The trained decision tree's rules are converted into a text-based hierarchy of nested "if-then" conditions. This verbalized rule, along with the explicit functional group counts, is appended directly to the LLM's training prompt.
  4. Test-Time Scaling (Rule-Consistency): Instead of relying on a single tree, the authors train a Random Forest ($N$ trees). During inference, they construct $N$ different prompts using $N$ different verbalized rules, generate parallel predictions from the LLM, and average the results. This acts as a text-based version of ensemble bagging.
Figure 2: An overview of our proposed method, consisting of 4 stages: ①Extracting FGs, ②Building Specialist Models, ③Distilling Knowledge to LLMs, and ④Test-Time Scaling.

Datasets

The method was evaluated on the Therapeutics Data Commons (TDC) benchmark, focusing on Molecular Property Prediction. The evaluation spans 22 distinct tasks (9 regression and 13 classification tasks) covering various ADMET properties. All datasets utilize a strict scaffold split, meaning the test molecules are structurally distinct from the training set, mimicking real-world drug screening challenges.

Key Insights and Ideas

  • Symbolic to Verbalized Knowledge: Rather than traditional knowledge distillation (which matches raw output probabilities between neural networks), TreeKD translates symbolic, hard-coded logic (decision trees) into natural language that an LLM can natively digest.
  • LLMs are Smart Students, Not Blind Mimics: A key finding was that LLMs do not blindly follow the provided tree rules. When the underlying decision tree made an incorrect prediction, the LLM overrode the rule and predicted correctly roughly 75% to 78% of the time. The LLM successfully synthesizes its internal pre-trained knowledge with the prompt's context.
  • Conquering Activity Cliffs: TreeKD significantly improved the LLM's ability to correctly classify pairs of highly similar molecules with opposite properties, a classic weak spot for deep generalist models.

Key Results

The authors evaluated TreeKD using two 2B-parameter open-source models: Gemma-2-2B and Granite-3.3-2B.

  • Performance Boost: Compared to naive training (SMILES strings only), TreeKD improved predictive accuracy on 19 out of 22 tasks for both LLMs.
  • SOTA Comparison: TreeKD consistently outperformed TxGemma-2-2B, the previous state-of-the-art generalist model in its size class, on 19/22 tasks for Gemma and 18/22 tasks for Granite.
  • Ensembling Success: The proposed rule-consistency technique consistently outperformed standard self-consistency (sampling multiple times from the same prompt), proving that injecting structural rule diversity at test time yields more robust molecular predictions.

Limitations and Future Work

  • Prompt Overhead: Prepending long, hierarchical decision tree rules significantly increases prompt token length. This raises both training and inference computational costs, though the authors demonstrate that prefix-caching techniques can successfully mitigate inference latency.
  • Task Scaling: While 22 tasks cover standard benchmarks, real-world drug discovery pipelines often require evaluating hundreds of properties simultaneously. Scaling the pipeline to handle this complexity remains an open challenge.
  • Broader Modalities: The current framework is tailored strictly to small molecules and functional groups. Extending this tree-distillation approach to larger biological entities like proteins or nucleic acids is a clear direction for future research.

Summary updated

Abstract

Molecular Property Prediction (MPP) is a fundamental problem in drug discovery that has recently attracted growing attention. Large Language Models (LLMs), known for their impressive proficiency across domains, show promise as generalist models for MPP. However, their current performance remains below the threshold needed for practical adoption. To bridge this gap, we propose TreeKD for distilling the knowledge of tree-based specialist models into LLMs to complement the internal knowledge of LLMs and improve their predictive accuracy. For each property, we train a specialist decision tree using features derived from 40K functional groups in the input molecules. Then, the predictive rule learned by the decision tree, which encodes its knowledge, is verbalized and incorporated into the prompts for training LLMs. In addition, by replacing a single decision tree with a Random Forest, we introduce a test-time scaling technique called rule-consistency, which aggregates predictions generated from different prompts constructed with different rules. An extensive evaluation with two LLMs, Gemma-2-2B and Granite-3.3-2B, on the TDC benchmark with 22 prediction tasks shows that our method substantially enhances the performance of LLMs, advancing the development of generalist models for MPP.
BibTeX
@misc{le2026can,
  author = {Khiem Le and Sreejata Dey and Marcos Martínez Galindo and Vanessa Lopez and Ting Hua and Nitesh V. Chawla and Hoang Thanh Lam},
  title = {{Can Decision Trees Teach Large Language Models? Distilling Verbalized Knowledge for Molecular Property Prediction}},
  year = {2026},
  eprint = {2603.12344},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.12344},
}
Figure