Transformers and the Representation of Biomedical Background Knowledge

Oskar Wysocki, Zili Zhou, Paul O'Regan, Deborah Ferreira, Magdalena Wysocka, Dónal Landers, André Freitas

TL;DR

Researchers investigated whether specialized transformer models like BioBERT and BioMegatron truly "understand" biology or are simply sophisticated pattern matchers. By using probing and clustering techniques, they discovered that while these models encode significant biological knowledge—even grouping genes by signaling pathways—much of this internal semantic structure is actually lost or "corrupted" during the fine-tuning process for specific tasks.

What Problem Does This Paper Address?

In the high-stakes world of cancer precision medicine, AI models are increasingly used to interpret the clinical significance of genomic alterations. However, these models often act as "black boxes." We know they perform well on benchmarks, but we don't fully understand their internal representation of biomedical entities (genes, variants, drugs, diseases) and the relations between them.

The authors address a critical gap: the lack of interpretability and the risk of "shortcut learning." If a model predicts a drug-gene association correctly, is it because it understands the underlying biology, or because that drug simply appeared frequently in the training data? This paper peels back the layers of these models to see how biological background knowledge is stored and how it changes when the model is trained for a specific task.

Methods

The study compares three main architectures: BERT-base (general domain), BioBERT, and BioMegatron (biomedical domain). They evaluated these models on two primary tasks:

  1. Task 1 (Binary Relations): Recognizing if a pair (e.g., drug-gene) exists in a known database.
  2. Task 2 (N-ary Relations): Classifying the clinical significance (Sensitivity/Response vs. Resistance) of a quadruple: [variant]-[gene]-[disease]-[drug].

To look "under the hood," they employed:

  • Probing: They trained simple linear classifiers (probes) on top of frozen model embeddings to see if the embeddings contained enough information to identify an entity's type (e.g., "Is this word a drug?").
  • Unsupervised Clustering: Using UMAP for dimensionality reduction and HDBSCAN/HAC for clustering, they visualized how the models naturally group entities in high-dimensional space.
  • Bias Analysis: They compared performance on "imbalanced" vs. "balanced" datasets to quantify how much the models rely on statistical frequency versus actual biological logic.

Datasets

The researchers utilized the CIViC (Clinical Interpretation of Variants in Cancer) database, a high-quality, expert-curated knowledge base.

  • Task 1: Generated ~8,000 entity pairs (true associations vs. random negative samples).
  • Task 2: Generated ~3,000 quadruples describing clinical evidence.
  • Control: A K-Nearest Neighbors (KNN) baseline was used to represent a model with no pre-trained linguistic or biological knowledge, relying solely on entity distribution.

Key Insights and Ideas

  • Fine-tuning is a Double-Edged Sword: While fine-tuning makes a model better at a specific task (like classification), it significantly degrades the model's general "understanding" of entity types. The probing experiments showed that pre-trained models have very "pure" clusters of genes, drugs, and variants, but these clusters become messy and overlapping after fine-tuning.
  • Emergent Biological Logic: In pre-trained models, genes weren't just grouped because they were "genes"; they clustered according to biological function. For example, the models naturally grouped genes belonging to the PI3K/AKT/mTOR signaling pathway together without any explicit supervision.
  • The "Frequency Trap": The models are highly sensitive to data imbalance. If a specific drug appears in many "True" associations in the training set, the model develops a bias to label any pair containing that drug as "True," regardless of the biological reality. This highlights a major safety concern for clinical applications.
  • BioMegatron vs. BioBERT: BioMegatron (a larger model) appeared to retain more semantic information after fine-tuning compared to BioBERT, suggesting that larger architectures might be more robust at preserving background knowledge while learning new tasks.

Key Results

  • Task Performance: Both BioBERT and BioMegatron achieved high accuracy on Task 2 (AUC > 0.8), significantly outperforming the KNN baseline.
  • Probing Accuracy: Pre-trained BioBERT could identify entity types with ~94% accuracy. After fine-tuning for Task 1, this dropped to ~73%.
  • Semantic Homogeneity: In pre-trained models, clusters were nearly 100% homogeneous (containing only one entity type). In fine-tuned models, homogeneity dropped as the model began to project different entity types (like drugs and variants) into the same space to satisfy the specific classification goal.
  • Misclassification of "Famous" Facts: Surprisingly, even well-supported clinical facts (Level A evidence) were sometimes misclassified if the specific entities were underrepresented in the training slice, proving that "fame" in the literature doesn't always translate to accuracy in the model.

Limitations and Future Work

The authors acknowledge that their study is limited by the inherent biases in the CIViC database and the biomedical literature itself (where positive results are reported more often than negative ones).

Future Directions: * Developing fine-tuning techniques that "freeze" or protect the underlying semantic structure of the model to prevent knowledge corruption. * Integrating structured knowledge (like ontologies or knowledge graphs) directly into the transformer architecture to provide a "safety rail" against statistical biases. * Expanding the analysis to more complex n-ary relations and larger, more diverse biomedical corpora.

Summary updated

BibTeX
@article{wysocki2023transformers,
  author = {Oskar Wysocki and Zili Zhou and Paul O'Regan and Deborah Ferreira and Magdalena Wysocka and Dónal Landers and André Freitas},
  title = {{Transformers and the Representation of Biomedical Background Knowledge}},
  journal = {Comput. Linguistics},
  year = {2023},
  doi = {10.1162/coli\_a\_00462},
}
Figure