Neural Architecture Search for Joint Optimization of Predictive Power and Biological Knowledge

Zijun Zhang, Linqi Zhou, Liangke Gou, Ying Nian WuarXiv 2019 · OpenReview

Abstract

We report a neural architecture search framework, BioNAS, that is tailored for biomedical researchers to easily build, evaluate, and uncover novel knowledge from interpretable deep learning models. The introduction of knowledge dissimilarity functions in BioNAS enables the joint optimization of predictive power and biological knowledge through searching architectures in a model space. By optimizing the consistency with existing knowledge, we demonstrate that BioNAS optimal models reveal novel knowledge in both simulated data and in real data of functional genomics. BioNAS provides a useful tool for domain experts to inject their prior belief into automated machine learning and therefore making deep learning easily accessible to practitioners. BioNAS is available at https://github.com/zj-zhang/BioNAS-pub.

Introduction

Deep learning has revolutionized genomics, enabling high-accuracy predictions of DNA and RNA binding sites. However, these models often function as "black boxes," making it difficult for researchers to trust or interpret their findings in a clinical context. Designing architectures that are both predictive and biologically interpretable typically requires extensive machine learning expertise. BioNAS addresses this by automating the search for neural network architectures that are inherently consistent with existing biological knowledge.

The BioNAS Framework

BioNAS integrates biological priors directly into the Neural Architecture Search (NAS) process. Unlike traditional NAS, which optimizes solely for validation accuracy, BioNAS employs a dual-objective reward function.

The BioNAS framework overview. (a) The optimization loop incorporating empirical data and biological knowledge. (b) The target network architecture. (c) Conceptual landscapes of accuracy versus joint accuracy and knowledge optimization.

Knowledge Dissimilarity

The core innovation is the Knowledge Dissimilarity Function. This function quantifies the distance between a model's learned convolutional filters and known biological motifs (e.g., Position Weight Matrices). By minimizing this dissimilarity alongside prediction loss, the framework encourages the discovery of architectures that "speak the language" of biology.

Optimization Strategy

BioNAS utilizes a Recurrent Neural Network (RNN) controller to sample potential architectures. These "child networks" are evaluated, and the controller is updated using Proximal Policy Optimization (PPO). PPO ensures stable updates even with the high-variance reward signals inherent in biological data.

Results and Discussion

The effectiveness of BioNAS was evaluated on both simulated and real-world datasets.

Balancing Accuracy and Interpretability

A critical finding is that high predictive accuracy does not necessitate biological interpretability. Multiple architectures can achieve state-of-the-art performance while learning nonsensical internal representations. BioNAS allows researchers to navigate this trade-off by adjusting the "knowledge weight" ($\lambda$).

Dynamics of architecture search. (a) Relationship between $\lambda$ and model value/rank. (b, c) Evolution of layer type weights during search for different $\lambda$ values in Layer 1 and Layer 3.

As shown in the search dynamics, increasing the influence of biological knowledge shifts the controller's preference toward specific layer types that better capture known motifs without compromising the overall loss.

Discovering Novel Biology

In real-world eCLIP data, BioNAS searched over a million possible architectures. The optimal model successfully matched known RNA-binding motifs for proteins such as U2AF2, QKI, and RBFOX2. Crucially, for U2AF2, the model discovered a 3'-splice site pattern (AG) flanking the known motif—a feature absent from in vitro priors but biologically essential in the cellular environment. This demonstrates that BioNAS can anchor a model using incomplete knowledge to uncover missing in vivo context.

Conclusion

BioNAS provides a robust bridge between automated machine learning and domain-specific expertise. By treating biological knowledge as a regularizer during architecture search, it enables the creation of models that are both powerful and transparent, facilitating the discovery of novel regulatory mechanisms in functional genomics.

Summary updated

Abstract

We report a neural architecture search framework, BioNAS, that is tailored for biomedical researchers to easily build, evaluate, and uncover novel knowledge from interpretable deep learning models. The introduction of knowledge dissimilarity functions in BioNAS enables the joint optimization of predictive power and biological knowledge through searching architectures in a model space. By optimizing the consistency with existing knowledge, we demonstrate that BioNAS optimal models reveal novel knowledge in both simulated data and in real data of functional genomics. BioNAS provides a useful tool for domain experts to inject their prior belief into automated machine learning and therefore making deep learning easily accessible to practitioners. BioNAS is available at https://github.com/zj-zhang/BioNAS-pub.
BibTeX
@misc{zhang2019neural,
  author = {Zijun Zhang and Linqi Zhou and Liangke Gou and Ying Nian Wu},
  title = {{Neural Architecture Search for Joint Optimization of Predictive Power and Biological Knowledge}},
  year = {2019},
  eprint = {1909.00337},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.00337},
}
Figure