Bioformer-RetroMAE: retrieval-oriented pretraining of a compact biomedical language model for literature search

Jialu Li, Li FangarXiv 2026 · arXiv

TL;DR

Researchers have developed Bioformer-RetroMAE, a compact biomedical language model that outperforms much larger counterparts like BioBERT in literature retrieval. By using a specialized "retrieval-oriented" pretraining strategy, this 43-million-parameter model achieves superior accuracy while being three times faster and significantly cheaper to run than standard BERT-based models.

What Problem Does This Paper Address?

In the era of Retrieval-Augmented Generation (RAG), the speed and quality of document retrieval are critical. When you ask a biomedical AI a question, it must first search through millions of PubMed articles to find relevant evidence.

Most current systems use "bi-encoders" based on the standard BERT architecture (110M parameters). However, these models face two major hurdles:

  1. Efficiency: Embedding 40+ million PubMed citations is computationally expensive, and encoding user queries in real-time creates a latency bottleneck.
  2. Training Mismatch: Standard BERT is trained using Masked Language Modeling (MLM), which focuses on predicting individual words. It doesn't explicitly train the "summary vector" (the [CLS] token) that bi-encoders rely on to represent the meaning of an entire document.

The authors sought to create a model that is small enough for low-latency applications but smart enough to capture deep semantic meaning for complex biomedical queries.

Methods

The authors built Bioformer-RetroMAE in two distinct stages:

  1. RetroMAE Continual Pretraining: They started with Bioformer-8L, a compact model with only 8 layers and a hidden size of 512 (compared to BERT’s 12 layers and 768 hidden size). They then applied RetroMAE, a pretraining framework designed for retrieval. In this setup, the encoder sees a moderately masked text (25%), and its output [CLS] vector is sent to a tiny, one-layer decoder. This decoder must reconstruct a heavily masked version (50%) of the same text using only that [CLS] vector. This forces the model to pack as much semantic information as possible into that single vector.
  2. Bi-encoder Fine-tuning: After pretraining on 32 million PubMed records, the model was fine-tuned as a bi-encoder. They used "hard negatives"—documents that look relevant to a keyword search (BM25) but aren't actually answers—to teach the model to make finer semantic distinctions.

Datasets

  • Pretraining: Approximately 32 million PubMed titles and abstracts published through August 2026.
  • Evaluation: The BioASQ Task 12B benchmark (2024 edition). This includes 5,049 training questions and 340 "golden" test questions written by biomedical experts. The task requires retrieving relevant articles from the entire PubMed corpus.

Key Insights and Ideas

  • The [CLS] Bottleneck is a Feature: By forcing a one-layer decoder to reconstruct text from a single vector, RetroMAE effectively "compresses" the essence of a document. This fixes the primary weakness of standard MLM pretraining for retrieval tasks.
  • Size Isn't Everything: The 43M-parameter Bioformer-RetroMAE produces 512-dimensional vectors. This reduces the memory footprint of the search index by one-third compared to the 768-dimensional vectors produced by standard BERT models, all while achieving higher accuracy.
  • Hybrid is Still King: Despite the power of dense retrieval (searching by meaning), traditional lexical retrieval (searching by exact keywords via BM25) remains incredibly strong in biomedicine due to specific gene and drug names. The best results come from "Hybrid Retrieval," where Bioformer-RetroMAE and BM25 results are fused together.

Key Results

  • Superior Accuracy: Bioformer-RetroMAE outperformed the much larger PubMedBERT by 37.0% in Mean Average Precision (MAP) and 29.1% in Recall@100.
  • Massive Gains from RetroMAE: Compared to the original Bioformer (trained only with MLM), the RetroMAE pretraining improved MAP by 53.6% without increasing the model's size or inference time.
  • Efficiency: The model is roughly 40% the size of BioBERT and runs 3 times faster at inference.
  • Hybrid Performance: Fusing Bioformer-RetroMAE with BM25 improved the NDCG@10 score by 4.6% over using BM25 alone, a larger gain than that provided by fusing with larger models.

Limitations and Future Work

  • Benchmark Bias: The authors note that BioASQ questions are written by experts and often favor keyword matching (lexical retrieval). They suggest evaluating the model on more "natural language" clinical questions or RAG-specific pipelines in the future.
  • Baseline Comparisons: The study did not include MedCPT as a baseline because its training data (large-scale search logs) is not publicly available for a controlled comparison.
  • Distillation: Future work could involve using a "cross-encoder" (a more powerful but slower model) as a teacher to further refine the bi-encoder's performance through knowledge distillation.

Summary updated

Abstract

Dense retrieval is increasingly used for biomedical literature search and retrieval-augmented generation (RAG). Most biomedical bi-encoders are built on BERTBase-sized encoders pretrained with masked language modeling (MLM). These encoders are costly for corpus-scale indexing and latency-sensitive query encoding, and MLM does not explicitly train the [CLS] vector that a bi-encoder uses to represent a whole text. We present Bioformer-RetroMAE, a compact (43M-parameter) retrieval-oriented biomedical encoder obtained by continually pretraining Bioformer8L} with RetroMAE on approximately 32 million PubMed titles and abstracts. After bi-encoder fine-tuning with BM25 hard negatives, we evaluated it on BioASQ Task 12B test questions by retrieving from all PubMed abstracts. Bioformer-RetroMAE outperformed BioBERT and PubMedBERT bi-encoders fine-tuned under identical settings on all metrics, despite having about 40% of their parameters and producing 512-dimensional rather than 768-dimensional vectors. Compared with PubMedBERT, it improved mean average precision (MAP) by 37.0% and Recall@100 by 29.1%. Compared with the original Bioformer8L}, RetroMAE pretraining improved MAP by 53.6% and Recall@100 by 32.8% without changing model size or inference cost. In hybrid retrieval, fusing Bioformer-RetroMAE with BM25 through weighted reciprocal rank fusion improved MAP and NDCG@10 over BM25 alone by 4.9% and 4.6%, respectively, larger gains than those obtained by fusing BM25 with PubMedBERT or BioBERT.
BibTeX
@misc{li2026bioformer,
  author = {Jialu Li and Li Fang},
  title = {{Bioformer-RetroMAE: retrieval-oriented pretraining of a compact biomedical language model for literature search}},
  year = {2026},
  eprint = {2611.19862},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19862},
}
Figure