When a drug turns out to be harmful, the first signals usually sit in passive reporting systems: databases like the FDA's Adverse Event Reporting System, where a report exists only if a patient got sick enough to seek care and someone bothered to file it. This creates a well-known distortion — passive surveillance bias — where what the database records is driven as much by reporting behavior, prescription volume, and media attention as by true toxicity. This proposal argues that a large, ignored part of that distortion is chemical: drugs with similar molecular structures get prescribed, monitored, and reported on in correlated ways, and no existing bias-correction method uses molecular structure to explain and remove reporting artifacts. Meanwhile, the machine-learning tools needed to exploit this are mature but disconnected. Sequence labelers extract drug mentions from messy clinical text; chemoinformatics pipelines turn names into canonical molecular structures and fingerprints; tree ensembles predict toxicity from those fingerprints. What is missing is the pipeline that stitches them together and a debiasing model that sits at the join.
The core claim: if we standardize every drug mention in a passive surveillance database to a canonical molecular structure, compute structural fingerprints, and use those fingerprints as covariates in an ensemble-tree model of reporting propensity — then fuse the debiased passive labels with a small amount of clean experimental reference data through shrinkage estimation at tree leaves — we can recover adverse-drug-reaction risk rankings that are closer to experimental ground truth than any pipeline that either ignores structure or ignores reporting bias. Chemical structure standardization is the enabling mechanism; the BiLSTM-CRF extractor is the front end that makes the database structure-normalizable at scale; the ensemble tree is the model class that handles mixed molecular and epidemiological features while admitting the shrinkage-based bias correction; and passive surveillance bias is both the target problem and the evaluation regime.
Background and the Gap
Three literatures meet here without touching. First, chemical information extraction: BiLSTM-CRF models — a bidirectional recurrent network that reads a sentence in both directions, with a layer that decodes a globally consistent label sequence — extract drug names and adverse events from clinical text with high accuracy, ranking first in the MADE1.0 challenge with an overall F1 of 0.8413, including 0.9042 on drug names Bidirectional LSTM-CRF for Adverse Drug Event Tagging in Electronic Health Records, and 92.67% micro-averaged F1 on hospital discharge summaries GNTeam at 2018 n2c2: Feature-augmented BiLSTM-CRF for drug-related entity recognition in hospital discharge summaries. Comparable systems normalize chemical mentions, but to ontology codes — MeSH identifiers in one end-to-end system HITSZ_CDR: an end-to-end chemical and disease relation extraction system for BioCreative V, RxNorm concepts in the curated FAERS benchmark AERS-DM, where 74% of 1.5 million verbatim drug names were mapped to 14,489 RxNorm concepts using the rule-based MedEx tool Standardizing adverse drug event reporting data. None of these pipelines go further to canonical molecular structures — canonical SMILES or circular fingerprints — so downstream models cannot see that two differently named drugs are close chemical cousins.
Second, chemoinformatics: Random Forests on Morgan fingerprints (bit vectors encoding local molecular neighborhoods) predict in vitro toxicity well on curated assay data, reaching ROC-AUCs of 0.756–0.803 on individual Tox21 assays and at least 0.7 across all twelve Predictive Toxicology: Modeling Chemical Induced Toxicological Response Combining Circular Fingerprints with Random Forest and Support Vector Machine, and they stay well calibrated even under label sparsity, where a deep alternative reported ROC-AUC of 0.83 on SIDER while Random Forests matched its calibration curves Calibrated prediction of scarce adverse drug reaction labels with conditional neural processes. But these models are trained on clean, curated assay data; nobody has shown they survive contact with passive-report labels.
Third, surveillance-bias correction: epidemiologists correct passive-report distortions with aggregate models — back-calculation showed Zika circulated silently in Brazil for 12–15 months before detection On the origin and timing of Zika virus introduction in Brazil; a conditional probability-tree model showed that raw Australian chlamydia notifications misled a naive rate calculation into seeing falling incidence when true incidence was rising Estimating incidence of sexually transmissible infections in Australia; and a queueing-network sequential detector bounds false alarms when monitoring multiple correlated adverse events Active Postmarketing Drug Surveillance for Multiple Adverse Events. All three operate on patient counts and screening rates. None uses molecular structure, and the queueing model explicitly assumes a single drug with no chemical covariates. This is the gap: no framework uses molecular descriptors to explain why reporting rates vary across drugs, and no molecular-prediction framework treats its passive-report training labels as biased observations rather than ground truth.
A fourth result constrains the design rather than motivating it: simply pooling large biased observational data with small clean reference data makes error explode as confounding grows, whereas a tree ensemble with adaptive shrinkage at the leaves — fusing biased and unbiased sources with the bias penalized during splitting — prevented that failure FAST: a Fused and Accurate Shrinkage Tree for Heterogeneous Treatment Effects Estimation. That paper tested only on synthetic and educational benchmarks; we port its mechanism to pharmacovigilance, which is precisely the setting it was never applied to.
Core Claim
The proposal's central hypothesis is that reporting propensity in spontaneous adverse-event databases is partially predictable from chemical structure — structurally similar drugs share therapeutic classes, launch timing, prescriber populations, media coverage, and Weber-effect dynamics — so structural fingerprints carry information that identifies and removes ascertainment artifacts that aggregate volume normalization misses. Formally, for a drug–event pair with observed report count $y$, we model the expected count under pure reporting behavior as a function of molecular fingerprint $x$, exposure proxies, and time since launch, and we estimate the residual biological signal as what remains after the reporting model is subtracted, shrunk toward a small clean reference set in the manner of the fused shrinkage tree FAST: a Fused and Accurate Shrinkage Tree for Heterogeneous Treatment Effects Estimation. The decisive prediction: predictions from the debiased model should agree with held-out experimental labels (Tox21, SIDER) substantially better than predictions from the same trees trained on uncorrected report counts, and better than a volume-normalized disproportionality baseline.
Research Program
The build has three stages. Stage one is extraction and standardization: run a BiLSTM-CRF with character-level CNN embeddings over the verbatim drug-name fields and free-text portions of the AERS-DM corpus, reusing the dual-level character-plus-word design that won MADE1.0 Bidirectional LSTM-CRF for Adverse Drug Event Tagging in Electronic Health Records and gazetteer augmentation shown to lift difficult ADE classes GNTeam at 2018 n2c2: Feature-augmented BiLSTM-CRF for drug-related entity recognition in hospital discharge summaries. The extractor's job here is narrower than full NER: it must recover the ~26% of verbatim drug names that MedEx could not normalize in AERS-DM Standardizing adverse drug event reporting data — typos, trade names, abbreviations — because character-level models are exactly what handles those Comparing CNN and LSTM character-level embeddings in BiLSTM-CRF models for chemical and disease named entity recognition. Each recovered name is then passed through a chemoinformatics standardization pipeline — salt stripping, charge and tautomer normalization, canonical SMILES, then ECFP fingerprints — following the curation recipe validated on toxicology data Predictive Toxicology: Modeling Chemical Induced Toxicological Response Combining Circular Fingerprints with Random Forest and Support Vector Machine and the rule sets used at scale in chemogenomics databases ExCAPE-DB: an integrated large scale dataset facilitating Big Data analysis in chemogenomics. The output measure of stage one is coverage: the fraction of reports attached to a canonical structure, which we compare against AERS-DM's 74% RxNorm coverage.
Stage two is the reporting-propensity model: a Random Forest trained to predict report volume per drug–event–time cell from fingerprints, drug class, time since launch (the Weber effect), and exposure proxies, following the epidemiological insight that the pathway from latent event to filed report is a sequence of conditional steps whose rates vary systematically across subgroups Estimating incidence of sexually transmissible infections in Australia. Crucially, the fingerprint features let the model pool information across structurally related drugs — a drug with no marketing history of its own inherits a reporting prior from its chemical neighbors. We verify this pooling is real by permuting the fingerprint features and measuring how much reporting-rate predictability collapses.
Stage three is fused debiased prediction: a shrinkage tree ensemble (rfFAST-style FAST: a Fused and Accurate Shrinkage Tree for Heterogeneous Treatment Effects Estimation) that takes the debiased passive signals as biased observational inputs and the Tox21 assay outcomes — and, where overlapping, SIDER — as the small clean reference set, shrinking leaf estimates toward the reference and penalizing splits into regions where the observational data is most confounded. The deliverable is a per-drug, per-event risk score with calibrated probabilities, exploiting the documented calibration strength of tree ensembles on fingerprint data Calibrated prediction of scarce adverse drug reaction labels with conditional neural processes.
One design tension must be honored, not ignored: an aggressive multi-step normalization pipeline was shown to degrade downstream relation extraction compared with simpler direct mapping in the HITSZ_CDR system HITSZ_CDR: an end-to-end chemical and disease relation extraction system for BioCreative V, while chemoinformatics work shows the opposite — non-canonical representations actively damage downstream models Chemical Language Models Have Problems with Chemistry: A Case Study on Molecule Captioning Task. We resolve this empirically by carrying both representations: the raw verbatim text as categorical features for the extractor and reporting model, and the canonical structure only for the fingerprint features, never collapsing one into the other.
Baselines
The do-nothing reference point is raw disproportionality scoring on the AERS-DM benchmark: report counts per drug–event pair, optionally divided by a prescription-volume proxy, with no structure, no extraction, and no bias modeling. This is what pharmacovigilance practice roughly does, and the chlamydia study showed precisely this kind of naive normalization fails — dividing notifications by tests suggested falling incidence when the probability-tree model showed true incidence rising from 1,800 to 3,200 per 100,000 Estimating incidence of sexually transmissible infections in Australia. Beating it means producing risk rankings with higher agreement with held-out experimental labels.
The strongest existing end-to-end competitor is the AERS-DM standardization pipeline itself Standardizing adverse drug event reporting data: rule-based MedEx normalization (F-measure 95.5% on its test fields), RxNorm and MedDRA mapping, and disproportionality analysis on the resulting 37 million drug–ADE records. We will run it on the identical data slice, and beat it only if our structure-aware pipeline both recovers more drug mentions (coverage above its 74%) and produces debiased risk scores with measurably higher agreement with Tox21 and SIDER held-out labels. Fairness requires the same input records, the same deduplication, and the same tuning budget per method.
The strongest machine-learning competitor is a Random Forest on ECFP fingerprints trained directly on raw (uncorrected) FAERS labels — the natural extension of the predictive-toxicology recipe Predictive Toxicology: Modeling Chemical Induced Toxicological Response Combining Circular Fingerprints with Random Forest and Support Vector Machine to surveillance data, and exactly the pipeline Gap 2 in the literature warns will conflate reporting volume with toxicity. It gets identical fingerprints, identical hyperparameter search budget, and identical compute. Beating it isolates the value of debiasing, since the only difference is the correction stage. Its reference performance on clean data is known: ROC-AUC at or above 0.7 on all twelve Tox21 assays, peaking at 0.803 on NR.PPAR_gamma Predictive Toxicology: Modeling Chemical Induced Toxicological Response Combining Circular Fingerprints with Random Forest and Support Vector Machine; we expect its uncorrected variant to degrade against held-out labels, and that degradation is itself a result. For calibration comparisons on SIDER, the reported number to beat or match is the conditional-neural-process ROC-AUC of 0.83 under both complete and 50%-sparse labels, with Random Forests already matching its calibration Calibrated prediction of scarce adverse drug reaction labels with conditional neural processes.
The key ablation removes only the structural covariates from the reporting-propensity model, replacing fingerprints with RxNorm-class one-hot encodings, so the reporting model can pool across therapeutic classes but not across chemical neighborhoods. If the fingerprint version does not beat this ablation, the central hypothesis is wrong and the proposal should be abandoned rather than rescued.
Why not just…?
Why not just fine-tune a transformer or LLM for the extraction stage? The extraction stage is instrumental, not the contribution, and the BiLSTM-CRF is chosen on evidence, not nostalgia: the dual-level BiLSTM-CRF won MADE1.0 at 0.8413 F1 Bidirectional LSTM-CRF for Adverse Drug Event Tagging in Electronic Health Records, and feature-augmented BiLSTM-CRF reached 92.67% on n2c2 GNTeam at 2018 n2c2: Feature-augmented BiLSTM-CRF for drug-related entity recognition in hospital discharge summaries. More importantly, general language models demonstrably mishandle chemical representations — they output inconsistent properties when the same molecule is written in different but equivalent strings Chemical Language Models Have Problems with Chemistry: A Case Study on Molecule Captioning Task — and chemistry-tuned LLMs only became usable after heavy instruction tuning on canonicalized data LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset. That said, this is a fair threat to the extraction stage only; if a transformer extractor raises drug-name coverage beyond our BiLSTM-CRF, we swap it in — the proposal's claim lives in the debiasing stage, not the tagger. We will run one transformer extractor as an additional baseline and report the coverage difference honestly.
Why not just use MedEx and the existing AERS-DM pipeline? MedEx performs well on its own test fields (F-measure 95.5%) but left 26% of unique drug names unnormalized, could not accommodate vocabulary updates, and struggled with foreign brand names and ambiguous free text Standardizing adverse drug event reporting data. Rule-based clinical tools alone also proved insufficient on hard classes — CLAMP/cTAKES outputs only helped when used as features inside a learned model, lifting ADE F1 from 28.17% to 40.50% GNTeam at 2018 n2c2: Feature-augmented BiLSTM-CRF for drug-related entity recognition in hospital discharge summaries. More fundamentally, MedEx normalizes to ontology codes, not structures, so it cannot supply the fingerprints the debiasing model needs. AERS-DM is therefore our data foundation and strongest baseline, not a substitute.
Why not just divide report counts by prescription volume? This is the naive-disproportionality baseline, and the evidence against it is direct: the Australian STI study showed that dividing notifications by tests produced a trend opposite to the truth, which only a conditional probability model over the full care-seeking and testing pathway corrected Estimating incidence of sexually transmissible infections in Australia. In pharmacovigilance the same logic applies with an added failure mode: reporting spikes follow drug launch and media attention (the Weber effect) Standardizing adverse drug event reporting data, which volume normalization cannot see at all. We include this baseline explicitly, and the decisive experiment below tests whether structure-aware correction beats it on held-out experimental labels.
Why not just use a deep model (GNN or neural process) instead of tree ensembles? On clean data the deep alternatives win only narrowly if at all: the conditional neural process reached ROC-AUC 0.83 on SIDER, but Random Forests matched its calibration and the deep model's advantage appeared only under engineered label sparsity, while XGBoost degraded and underestimated positive probabilities Calibrated prediction of scarce adverse drug reaction labels with conditional neural processes. The decisive reason is architectural: the bias correction requires fusing biased and unbiased data sources with shrinkage at leaf estimates and bias-penalized splitting, a mechanism that exists in the shrinkage-tree literature FAST: a Fused and Accurate Shrinkage Tree for Heterogeneous Treatment Effects Estimation and has no standard analogue in end-to-end deep molecular models. We will still run one fingerprint-based deep baseline to check we are not leaving accuracy on the table.
Proposed Experiments
Experiment one tests extraction and standardization coverage. On the AERS-DM corpus and a held-out sample of verbatim drug names, measure the fraction of unique names mapped to a canonical structure by our BiLSTM-CRF-plus-standardization pipeline, against the 74% RxNorm coverage of MedEx-based AERS-DM Standardizing adverse drug event reporting data. Extractor quality is validated on MADE1.0, where the reference result to match is overall F1 of 0.8413 Bidirectional LSTM-CRF for Adverse Drug Event Tagging in Electronic Health Records, and on the CDR corpus, where the reference for chemical NER is 91.94% F1 with CNN character embeddings Comparing CNN and LSTM character-level embeddings in BiLSTM-CRF models for chemical and disease named entity recognition.
Experiment two tests the structural-reportability hypothesis. Train the reporting-propensity forest on fingerprints plus launch time and exposure proxies; report its predictive power on held-out reporting cells, and the drop under fingerprint permutation. If fingerprints add nothing over therapeutic-class one-hots (the ablation), the central claim fails here, cheaply and early.
Experiment three is the decisive bias-correction comparison. Take Tox21 assay outcomes and SIDER ADR labels as held-out clean references for the overlapping drug sets; the reference numbers are Random-Forest ROC-AUC of at least 0.7 across all twelve Tox21 assays with a best of 0.803 Predictive Toxicology: Modeling Chemical Induced Toxicological Response Combining Circular Fingerprints with Random Forest and Support Vector Machine and CNP ROC-AUC of 0.83 on SIDER Calibrated prediction of scarce adverse drug reaction labels with conditional neural processes. Compare four pipelines on identical data and compute: raw disproportionality, MedEx/AERS-DM disproportionality, uncorrected Random Forest on fingerprints, and the fused shrinkage pipeline. Metrics are rank correlation and ROC-AUC against the clean references, plus calibration curves. The proposal succeeds if the fused pipeline beats all three baselines on agreement with Tox21/SIDER and if the uncorrected forest demonstrably degrades relative to its clean-data performance — confirming that passive labels poison naive molecular prediction and that the correction removes the poison.
Experiment four stress-tests the fusion under controlled confounding, in the spirit of the FAST evaluation, which reported no numeric metrics on Lacuna but demonstrated visually increasing advantage over generalized random forests and honest trees as bias increased FAST: a Fused and Accurate Shrinkage Tree for Heterogeneous Treatment Effects Estimation. We inject synthetic Weber-effect spikes and class-correlated reporting surges into the passive labels at graded intensities and track each pipeline's degradation curve; the fusion should degrade gracefully where naive pooling surges in error, as that paper's mechanism predicts. A fifth, secondary check evaluates whether the debiased scores improve sequential multi-event detection when fed to a queueing-network monitor in place of raw counts Active Postmarketing Drug Surveillance for Multiple Adverse Events, though this is exploratory since that system was validated only on simulated patient-flow data.
References
- Bidirectional LSTM-CRF for Adverse Drug Event Tagging in Electronic Health Records
- GNTeam at 2018 n2c2: Feature-augmented BiLSTM-CRF for drug-related entity recognition in hospital discharge summaries
- HITSZ_CDR: an end-to-end chemical and disease relation extraction system for BioCreative V
- Standardizing adverse drug event reporting data
- Predictive Toxicology: Modeling Chemical Induced Toxicological Response Combining Circular Fingerprints with Random Forest and Support Vector Machine
- Calibrated prediction of scarce adverse drug reaction labels with conditional neural processes
- On the origin and timing of Zika virus introduction in Brazil
- Estimating incidence of sexually transmissible infections in Australia
- Active Postmarketing Drug Surveillance for Multiple Adverse Events
- FAST: a Fused and Accurate Shrinkage Tree for Heterogeneous Treatment Effects Estimation
- Comparing CNN and LSTM character-level embeddings in BiLSTM-CRF models for chemical and disease named entity recognition
- ExCAPE-DB: an integrated large scale dataset facilitating Big Data analysis in chemogenomics
- Chemical Language Models Have Problems with Chemistry: A Case Study on Molecule Captioning Task
- LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset