Focal Distillation of Multi-View Molecular Models for Imbalanced, Low-Data Toxicity Prediction

Predicting whether a molecule is toxic is one of the highest-stakes and worst-conditioned problems in applied machine learning: the labeled datasets are small (hundreds to a few thousand compounds), the positives are rare (often 1–5% of samples), and the signal is scattered across fundamentally different descriptions of the same molecule — a text string of atoms, a 2D graph of bonds, a rendered 3D shape. Multi-view molecular models that fuse these descriptions are accurate but must run several large encoders at once, which is too slow for screening libraries of millions of compounds. Knowledge distillation — training a small "student" network to mimic the soft output probabilities and internal features of a big "teacher" — can compress the ensemble into one cheap model, but every existing molecular distillation pipeline weighs all training molecules equally, so on an imbalanced toxicity dataset the distillation signal is dominated by the easy, confidently-negative majority, and the student never learns what a rare toxic compound looks like.

This proposal combines the four directions with distinct roles: multi-view molecular representation supplies the teacher, knowledge distillation supplies the compression mechanism, the model-complexity trade-off defines both the student family and the evaluation regime (small, skewed datasets where capacity must be rationed), and focal loss supplies the missing ingredient — a difficulty-dependent weighting applied to the distillation objective itself. The core deliverable is a single cheap student that beats both deep multi-view ensembles and classical descriptor baselines on rare-class recall for imbalanced toxicity tasks.

Core Claim

The claim is that distillation is not imbalance-neutral. When a teacher's soft distribution for an easy negative is matched by the student, that sample contributes as much gradient as a hard positive, so the student's capacity is spent reproducing the majority class's probability landscape. Focal loss — the modulating factor $(1-p_t)^\gamma$ from Focal Loss for Dense Object Detection, where $p_t$ is the model's probability for the true class — was invented precisely to suppress easy examples, but it has only ever been applied to the student's supervised loss, never to the teacher-matching term. We propose to apply it to the distillation loss itself:

$$\mathcal{L}_{\text{focal-KD}} = -(1-p_t)^\gamma \; \tau^2 \, \mathrm{KL}\!\left(q^{T}_{\tau} \,\|\, q^{S}_{\tau}\right)$$

where $q^{T}_{\tau}$ and $q^{S}_{\tau}$ are temperature-softened teacher and student distributions at temperature $\tau$, and $p_t$ is the student's probability for the ground-truth class. The effect is that when the student already handles an easy negative, the pressure to copy the teacher on that molecule nearly vanishes; the remaining gradient budget concentrates on the rare, hard positives — exactly the samples a toxicity screening model cannot afford to miss. This is combined with a deliberately constrained student (a shallow network over fixed fingerprints plus a small graph encoder), because the complexity trade-off literature consistently shows that on small datasets, high-capacity architectures memorize noise while constrained ones generalize.

What Exists and What Is New

The closest existing work is IEM (An Image-enhanced Molecular Graph Representation Learning Framework), which pre-trains a visual teacher on two million rendered 3D conformations and distills it into 2D graph students, reaching 73.89% ROC-AUC averaged over MoleculeNet classification tasks when combined with GraphMVP. IEM establishes that cross-view distillation works and that the expensive view can be discarded at test time. But IEM uses uniform feature and prediction losses, never evaluates on skewed-low-data downstream splits, and only considers deep GNN students. Our proposal differs on all three axes: the distillation loss itself is focal-modulated for imbalance, the student family is swept from a logistic head on fingerprints up to a full GNN to locate the capacity crossover under distillation, and the evaluation regime is deliberately the one where IEM's protocol is weakest — small imbalanced toxicity endpoints.

Three other lines define the gap. Balancing Imbalanced Toxicity Models: Using MolBERT with Focal Loss shows frozen MolBERT embeddings plus weighted focal loss beat fine-tuning on drug-induced liver injury (balanced accuracy ~0.68 → 0.72, F1 ~0.37 → 0.40, ROC-AUC 0.70), but uses a single 1D string view and no distillation. Pre-training Molecular Graph Representation with 3D Geometry (GraphMVP) transfers 3D geometry into a 2D graph encoder during pre-training (73.07% average ROC-AUC on MoleculeNet) but leaves the student a standard GNN trained with unmodulated losses. And Accelerating Molecular Graph Neural Networks via Knowledge Distillation demonstrates feature-level distillation into a small PaiNN student (680 samples/sec versus the teacher's 107) for regression only — its 96.7% energy-gap recovery versus 62.5% force-gap recovery also warns that dense, high-order signals are hard to distill, which motivates our choice of scalar classification endpoints rather than geometric targets. In vision, Focal and Global Knowledge Distillation for Detectors already showed that focal ideas belong inside distillation (raising ResNet-50 Faster R-CNN mAP from 38.4% to 42.0% on COCO2017), but only as spatial foreground/background masks; nobody has carried difficulty-modulated distillation into molecular classification, where the imbalance is in labels rather than pixels.

Research Program

The first work package builds the multi-view teacher. We fuse three views with documented low information overlap — 1D SMILES via a pretrained transformer, 2D molecular graphs via a GNN, and 3D-rendered visual depictions via a convolutional encoder — following the evidence in An Image-enhanced Molecular Graph Representation Learning Framework that 2D and 3D graph features are highly correlated while graph and visual renderings are not, so naive concatenation wastes the non-redundant signal. The teacher is trained with standard multi-view alignment on large unlabeled corpora (PCQM4Mv2-style pre-training, as in GraphMVP's contrastive-plus-generative recipe) and then fine-tuned on each imbalanced endpoint. We deliberately do not innovate on the teacher; its job is only to be a strong, multi-perspective source of soft labels.

The second work package is the focal distillation objective. The student is trained jointly on ground-truth labels with weighted focal loss and on the teacher's softened distributions with the focal-modulated KL term above, plus an optional feature-alignment term for intermediate representations in the spirit of IEM's Knowledge Enhancer, also focal-modulated. We will analyze the gradient geometry: for the easy-negative bulk, the modulating factor at $\gamma=2$ shrinks the distillation gradient by roughly $(1-p_t)^2$, and we will verify empirically that the fraction of total distillation gradient coming from positive samples rises from near-zero (uniform KD) to a value matching their task importance. A known hazard, documented for focal loss generally, is that hard mislabeled compounds get maximal weight; we will therefore pair focal KD with label-noise robustness checks (confidence thresholds on the teacher, and comparison against AdaFocal: Calibration-aware Adaptive Focal Loss, which adapts $\gamma$ per sample to control miscalibration).

The third work package is the capacity study. We train a ladder of students on the same distilled signal: a weighted logistic regression over ECFP/MACCS fingerprints, a shallow MLP, a small GNN, and a full-size GNN, at downstream training-set sizes from 100 to full. The complexity-trade-off literature predicts a crossover curve — below a data threshold, the constrained student should win (Neural network language models for low resource languages observed exactly this crossover between feed-forward and recurrent models). The open question we test is whether focal distillation shifts that crossover point to the right, letting a small student exploit multi-view knowledge that its own capacity could never extract directly.

Proposed Experiments

Evaluation is on three imbalanced molecular benchmarks, chosen because each has published numbers we can compare against. First, the Tox21 held-out set of 647 compounds from Measuring AI Progress in Drug Discovery: A Reproducible Leaderboard for the Tox21 Challenge, where descriptor-based models from 2015 (DeepTox) and 2017 (Self-Normalizing Neural Networks) still top the leaderboard over modern GNNs; the file reports no exact AUC for DeepTox or SNN, but does report a zero-shot GPT-OSS 120B AUC of 0.703, a useful contemporary reference point. Second, the DILI dataset of 1,554 compounds across histopathology, blood-marker, and clinical-ADR endpoints from Balancing Imbalanced Toxicity Models: Using MolBERT with Focal Loss, where the strongest published configuration (frozen MolBERT + weighted focal loss) reports balanced accuracy 0.72, F1 0.40, and ROC-AUC 0.70. Third, the acute dermal toxicity (rabbit and rat) datasets from Integrating Multiview Information for Enhanced Deep Learning-Based Acute Dermal Toxicity Prediction, where the best multi-view system reports gains of up to 11.39% AUPRC and 15.07% recall over the second-best model on rabbit toxicity, and 3.43% AUPRC on rat. We also run the MoleculeNet scaffold-split classification suite (MoleculeNet: A Benchmark for Molecular Machine Learning) for comparability with IEM (GraphMVP+IEM: 73.89% average ROC-AUC; GraphMVP alone: 73.07%). Primary metrics are rare-class recall, balanced accuracy, AUPRC, and MCC — ROC-AUC alone can flatter a model that still misses most positives. Every comparison is run at matched compute and tuning budget (same hyperparameter search protocol, same number of random seeds, same student input representations), and we log student inference throughput to confirm the deployment payoff, using the 680-versus-107 samples/sec gap from Accelerating Molecular Graph Neural Networks via Knowledge Distillation as the yardstick for what a successful compression should look like.

Baselines

The naive reference point is a fingerprint classifier: ECFP/MACCS descriptors with class-weighted logistic regression, which needs no training of any encoder and sets the floor for any method claiming value from neural representations. The strongest existing-method baseline is the frozen-embedding-plus-weighted-focal-loss configuration of Balancing Imbalanced Toxicity Models: Using MolBERT with Focal Loss — it is the direct competitor because it embodies the same philosophy (constrained predictor, imbalance-aware loss, no fine-tuning instability) while skipping distillation entirely; we run it with our strongest single-view frozen embedding and with the fused multi-view teacher used as a frozen feature extractor, tuning its loss weights and heads with exactly the same search budget as our student. The second strong baseline is descriptor-based SNN/DeepTox-style models as characterized in the Tox21 leaderboard study, since they currently top that held-out set. The key ablation isolates the proposal's single ingredient: identical teacher, identical student architecture, identical data — with uniform KL distillation versus focal-modulated distillation, and with focal loss on the supervised term only versus on both terms. Any improvement must be attributable to the focal modulation of the distillation loss, not to the teacher, the student, or the supervised weighting. Beating the frozen-embedding baseline means a statistically significant gain in balanced accuracy and positive-class recall across the three imbalanced benchmarks at matched compute; matching it with a tenfold smaller inference footprint would also count as a limited success, and we will report both honestly.

Why not just…?

Why not just use fingerprints with XGBoost or an SNN and skip deep learning? This is a real threat: on the Tox21 held-out set, descriptor models from a decade ago still beat modern GNNs (Measuring AI Progress in Drug Discovery), and the file reports no number where a modern method surpassed them there. But on the dermal toxicity datasets, the multi-view MVIToxNet outperformed tree-based fingerprint baselines by up to 11.39% AUPRC and 15.07% recall (Integrating Multiview Information for Enhanced Deep Learning-Based Acute Dermal Toxicity Prediction), and MoleculeNet reports learnable representations winning on 11 of 17 datasets. Our experiments settle this directly: the fingerprint classifier is the floor baseline, and if it cannot be beaten even by the distilled student, that is a publishable negative result about distillation in this regime — we will report it rather than argue around it.

Why not just freeze a pretrained model and train a weighted focal head? This is the strongest alternative, and it already works: it raised balanced accuracy to 0.72 on DILI (Balancing Imbalanced Toxicity Models: Using MolBERT with Focal Loss), and Emb-GAM-style shallow heads on frozen embeddings match large predictors at 1,000× lower latency (Emb-GAM: an Interpretable and Efficient Predictor using Pre-trained Language Models). Its weakness is informational, not computational: a frozen single-view embedding is locked to one modality, and the DILI study itself found some endpoints unlearnable from 1D strings alone. Our proposal is designed to be tested against exactly this baseline with the teacher's views ablated one at a time; if distillation adds nothing over the best frozen single-view or even frozen fused embedding with focal loss, the hypothesis is wrong and the experiment will say so.

Why not just use weighted cross-entropy instead of focal loss? Weighted cross-entropy scales loss by class frequency but leaves easy majority negatives with full gradient mass per sample; focal loss suppresses each easy example's contribution by up to 99% at $\gamma=2$, and on DILI the peak numbers required the combination of class weighting and difficulty modulation, not weighting alone (Balancing Imbalanced Toxicity Models: Using MolBERT with Focal Loss). Within our own pipeline, the focal-versus-weighted ablation on the distillation term is precisely the experiment that answers this question, and we grant the honest caveat: $\gamma$ is a sensitive hyperparameter and can miscalibrate probabilities, which is why we include the adaptive variant (AdaFocal) as a check.

Why not just ensemble several multi-view models? Ensembling gives small gains on these datasets — up to 0.73% recall and 1.11% MCC on rabbit toxicity from weighted model averaging (Integrating Multiview Information) — but multiplies inference cost linearly in the number of models, which defeats the purpose for screening millions of compounds. Distillation collapses the ensemble into one student with no inference overhead, as IEM (image pipeline discarded at test time) and PaiNN-small (680 vs 107 samples/sec) both demonstrate.

Decisive Checks

The proposal lives or dies on three outcomes. First, the ablation must show that focal-modulated distillation, and not the teacher or the supervised focal loss alone, drives any rare-class recall gain over uniform KL distillation; if the two are statistically indistinguishable, the central claim is dead. Second, the capacity study must locate the crossover: we expect the distilled small student to beat the distilled full-size GNN below some training-set size and to beat its own from-scratch training everywhere; if distillation only helps large students on large datasets, it has nothing to say about the low-data regime this proposal targets. Third, on at least one of the three imbalanced benchmarks the distilled compact student must beat the frozen-embedding focal head at matched budget — otherwise the honest conclusion is that a strong frozen multi-view teacher is all one needs, and that too would be worth writing down.