DPCIPI: A pre-trained deep learning model for predicting cross-immunity between drifted strains of Influenza A/H3N2

Yiming Du, Zhuotian Li, Qian He, Thomas Wetere Tulu, Kei Hang Katie Chan, Lin Wang, Sen Pei, Zhanwei Du, Zhen Wang, Xiao-Ke Xu, Xiao Fan Liu

TL;DR

This paper introduces DPCIPI, a novel deep learning model that predicts cross-immunity between influenza strains using pre-trained gene sequence embeddings and a mutual information inference layer. DPCIPI significantly outperforms existing methods in predicting hemagglutination inhibition titers, offering a more efficient and data-light approach to understanding viral evolution and aiding vaccine development.

What Problem Does This Paper Address?

Seasonal influenza poses a significant global health threat, infecting billions annually and causing millions of deaths. Vaccination is the primary defense, but influenza viruses constantly evolve through "antigenic drift," evading vaccine-induced immunity. Predicting cross-immunity – how well antibodies from one strain protect against another – is crucial for effective vaccine design and public health surveillance. Traditional methods, like hemagglutination inhibition (HI) assays, are labor-intensive and time-consuming, especially given the vast number of circulating strains. While deep learning models like LSTMs and CNNs have shown promise in biological sequence analysis, they often struggle with limited lab data for training and can overlook crucial inter-gene relationships when simply concatenating sequences. This paper aims to address these limitations by developing a more data-efficient and accurate model for predicting cross-immunity.

Methods

The DPCIPI model introduces several key innovations:

  • k-mer Preprocessing: Influenza gene sequences are first converted into "k-mers" (short, overlapping DNA segments of length k, here k=6). This is followed by sequence alignment and deduplication to focus on unique variations and reduce redundancy.
  • DNABERT Pre-trained Embeddings: Instead of random initialization, DPCIPI leverages DNABERT, a pre-trained model for DNA sequences. This allows DPCIPI to utilize learned representations of k-mers, significantly improving performance even with limited influenza-specific data. For any k-mers not seen in DNABERT's training, a clever averaging strategy based on neighboring k-mers is employed.
  • Mutual Information Inference Layer: This novel layer fuses the embeddings from two gene sequences. It goes beyond simple concatenation by using operations like dot product and subtraction to capture both similarities and dissimilarities between sequences, effectively identifying subtle genetic variations that influence cross-immunity. This draws inspiration from natural language inference techniques.
  • BiLSTM Encoding: A Bidirectional Long Short-Term Memory (BiLSTM) network is used to further encode the k-mer embeddings, capturing sequential dependencies within each gene.
  • Classification: Finally, a Multi-Layer Perceptron (MLP) takes the fused representation from the mutual information layer and predicts the cross-immunity effect.

Datasets

The study utilizes the Virus Hemagglutination Inhibition Dataset (VHID), a revised dataset compiled from Smith et al. [12]. This dataset contains 2,472 hemagglutination inhibition (HI) titer outcomes from 240 reference viruses and 43 test viruses. Gene sequences were retrieved from the NCBI GenBank. The dataset was split into training (strains before 1995) and testing (strains after 1995) sets to ensure the model can predict cross-immunity for future strains based on historical data.

Key Insights and Ideas

  • Pre-trained Models are Powerful for Biological Sequences: The success of DNABERT embeddings highlights that models pre-trained on large biological datasets can be effectively transferred to specialized tasks like influenza cross-immunity prediction, even when the target data is scarce.
  • Beyond Concatenation: Mutual Information Matters: Simply concatenating gene sequences for input can obscure important differences. The mutual information inference layer is crucial for explicitly capturing the nuanced relationships and dissimilarities between viral strains, leading to better predictions.
  • Data Efficiency is Key: The limited availability of experimental HI titer data is a major bottleneck. DPCIPI's approach, combining pre-trained models with a smart inference layer, demonstrates a path towards building effective models with less reliance on extensive lab-generated datasets.
  • Focusing on Differences: The mutual information operator's ability to highlight differences between sequences is particularly important for understanding antigenic drift, where small genetic changes can lead to significant immune evasion.

Key Results

DPCIPI demonstrated superior performance across various metrics compared to baseline models (statistical methods like Logistic Regression, Perceptron, Decision Tree, and neural networks like NN, CNN, BiLSTM).

  • Binary Cross-Immunity Prediction: DPCIPI achieved an accuracy of 89.69% and a Weighted F1 score of 88.14%. It showed improvements of 1.58% in F1, 2.34% in precision, 1.57% in recall, and 1.57% in Accuracy over the best baseline (BiLSTM).
  • Multi-level Cross-Immunity Prediction: DPCIPI achieved an accuracy of 64.69% and a Weighted F1 score of 64.71%. It showed improvements of 2.12% in F1, 3.50% in precision, 2.19% in recall, and 2.19% in Accuracy over the best baseline (BiLSTM).
  • The study also found that using gene sequence embeddings directly with statistical models (e.g., LR-GSE) was less effective than using them within a deep learning architecture like DPCIPI.

Limitations and Future Work

  • Fragmented Gene Sequences: The current DNABERT model has limitations on input sequence length, meaning it was trained on fragmented gene sequences. This can lead to incomplete information.
  • Future Work: The authors suggest training larger DNA length models with complete gene sequences to further enhance performance. They also envision DPCIPI's significant impact on vaccine development, disease surveillance, and preparedness strategies.

Summary updated

Abstract

Predicting cross-immunity between viral strains is vital for public health surveillance and vaccine development. Traditional neural network methods, such as BiLSTM, could be ineffective due to the lack of lab data for model training and the overshadowing of crucial features within sequence concatenation. The current work proposes a less data-consuming model incorporating a pre-trained gene sequence model and a mutual information inference operator. Our methodology utilizes gene alignment and deduplication algorithms to preprocess gene sequences, enhancing the model’s capacity to discern and focus on distinctions among input gene pairs. The model, i.e., DNA Pretrained Cross-Immunity Protection Inference model (DPCIPI), outperforms state-of-the-art (SOTA) models in predicting hemagglutination inhibition titer from influenza viral gene sequences only. Improvement in binary cross-immunity prediction is 1.58% in F1, 2.34% in precision, 1.57% in recall, and 1.57% in Accuracy. For multilevel cross-immunity improvements, the improvement is 2.12% in F1, 3.50% in precision, 2.19% in recall, and 2.19% in Accuracy. Our study showcases the potential of pre-trained gene models to improve predictions of antigenic variation and cross-immunity. With expanding gene data and advancements in pre-trained models, this approach promises significant impacts on vaccine development and public health.
BibTeX
@article{du2025dpcipi,
  author = {Yiming Du and Zhuotian Li and Qian He and Thomas Wetere Tulu and Kei Hang Katie Chan and Lin Wang and Sen Pei and Zhanwei Du and Zhen Wang and Xiao-Ke Xu and Xiao Fan Liu},
  title = {{DPCIPI: A pre-trained deep learning model for predicting cross-immunity between drifted strains of Influenza A/H3N2}},
  journal = {Journal of Automation and Intelligence},
  year = {2025},
  doi = {10.1016/j.jai.2025.03.004},
}
Figure