Enhancing Transcription Factor Prediction through Multi-Task Learning (Student Abstract)

Liyuan Gao, Matthew Zhang, Victor S. ShengAAAI 2024 · DOI

Introduction

Transcription factors (TFs) are the "master switches" of gene expression, binding to specific DNA sequences to regulate the transcription of genetic information from DNA to messenger RNA. Identifying these proteins from amino acid sequences is a fundamental challenge in molecular biology and drug discovery. Traditional computational methods, such as Support Vector Machines (SVMs), often struggle with the scale and complexity of modern biological datasets. While deep learning tools like DeepTFactor have improved performance, they often treat TF prediction as an isolated task, overlooking the valuable structural information contained within specific protein domains, such as Zinc Fingers, that are characteristic of TF activity.

Methodology

This research introduces a novel framework that combines state-of-the-art Natural Language Processing (NLP) with Multi-Task Learning (MTL) to improve prediction accuracy.

Protein Language Modeling and Compression

Instead of traditional "one-hot" encoding, the model utilizes ESM-2, a pre-trained protein language model. ESM-2 captures the "grammar" of protein sequences, providing high-dimensional semantic representations. To manage computational complexity for long sequences, the researchers implemented a sliding window technique for representation compression, averaging amino acid representations within fixed windows while retaining essential structural features.

Multi-Task Architecture

The core of the system is a shared neural network based on 1-D Convolutional Neural Networks (CNNs). This architecture branches into two distinct classification heads:

  1. Task A (TF Prediction): A binary classification task to determine if a protein is a transcription factor.
  2. Task B (Domain Classification): A multi-class classification task identifying specific TF-related structural domains.
The multi-task learning framework pipeline, integrating ESM-2 language modeling with a dual-head CNN architecture.

By optimizing both tasks simultaneously, the model leverages the synergy between domain recognition and general TF identification, allowing the network to learn nuanced patterns that single-task models might overlook.

Datasets

The study utilized data from UniProt, comprising 6,369 TF sequences and 5,000 Non-TF (NTF) sequences. For the domain classification task, TF sequences were categorized into five groups: Zinc Finger (ZF), Homeodomain (Homeo), Leucine Zipper (LZ), Other-TFs, and Non-TFs.

Key Results

The multi-task framework demonstrated superior performance across all evaluated metrics compared to both its single-task counterpart and established specialized tools.

  • TF Prediction Performance: The multi-task model achieved a peak accuracy of 94.32% and an F1 score of 0.9422. This significantly outperforms DeepTFactor (91.69% accuracy) and the older TFpredict (78.60% accuracy).
  • Domain Classification: The multi-task approach reached 90.85% accuracy in identifying specific domains, compared to 88.42% for the single-task version, confirming that joint optimization improves feature extraction for both tasks.
Performance comparison of the proposed multi-task model against existing benchmarks for transcription factor prediction.

Discussion and Future Work

The success of this framework highlights the power of transfer learning in biology. Using ESM-2 provides a significant "head start," reducing the volume of task-specific labeled data required. Furthermore, the synergy between domain classification and TF prediction suggests that functional prediction is most effective when informed by structural context.

Future research will focus on improving model interpretability to identify specific regulatory motifs and expanding the framework to include a broader range of rare TF domains. Additionally, exploring more lightweight versions of protein language models could facilitate high-throughput screening in resource-constrained environments.

Summary updated

BibTeX
@inproceedings{gao2024enhancing,
  author = {Liyuan Gao and Matthew Zhang and Victor S. Sheng},
  title = {{Enhancing Transcription Factor Prediction through Multi-Task Learning (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30446},
}
Figure