EpiRL: A Reinforcement Learning Agent to Facilitate Epistasis Detection

Kexin Huang, Rodrigo Frassetto NogueiraPrecision Health and Medicine 2020 · OpenReview

TL;DR

This paper introduces EpiRL, a novel reinforcement learning (RL) agent designed to efficiently detect gene-gene interactions (epistasis) crucial for understanding complex diseases. By framing epistasis detection as a one-step Markov Decision Process, EpiRL overcomes the computational intractability of exhaustive search methods, offering a more scalable approach to identifying these critical genetic relationships.

What Problem Does This Paper Address?

Understanding how genes interact (epistasis) is vital for explaining complex diseases and traits that single-gene analyses fail to capture. Traditional methods for detecting epistasis, like exhaustive combinatorial searches, become computationally infeasible as the number of genes (SNPs) increases. While some machine learning approaches exist, they often struggle with the sheer scale of genomic data or are limited to detecting interactions of a fixed, pre-determined size (e.g., 2-locus or 3-locus). This paper addresses the challenge of efficiently identifying epistatic interactions in large-scale genomic datasets without being constrained by a fixed interaction order.

Methods

The core innovation is modeling epistasis detection as a one-step Markov Decision Process (MDP).

  • State: A latent representation derived from genome data.
  • Action: Selecting individual SNPs.
  • Reward: A measure of the interaction strength of the selected SNPs, calculated using established metrics like Multifactor Dimensionality Reduction (MDR) Correct Classification Rate (CCR) and Rule Utility.

The authors employ a two-layer neural network to predict the probability of each SNP being part of a highly interacting set. An RL agent, specifically using the REINFORCE algorithm, learns a policy to select SNPs based on these probabilities. The agent is designed to select SNPs whose probabilities exceed a certain threshold, allowing for variable-sized interaction sets (from 2 to n-locus) in each iteration, thus avoiding the constraint of fixed interaction orders. The reward function combines CCR and Rule Utility to provide a robust measure of epistasis.

Datasets

The paper uses simulated data generated by the GAMETES software, which is designed to create random, pure, strict, n-locus epistasis models. For their preliminary study, they used a simulated 2-locus dataset with 600 sequences (half case, half control) and 100 SNPs, with specific parameters for heritability, prevalence, and minor allele frequency. The authors also mention plans to test on a real-world GWAS Coronary Artery Disease (CAD) dataset in future work.

Key Insights and Ideas

  • RL for Epistasis: Framing epistasis detection as an RL problem is a novel approach that elegantly handles the combinatorial explosion of gene interactions.
  • Variable Interaction Order: The method's ability to discover interactions of varying sizes (not fixed to 2 or 3 loci) is a significant advantage, as the true order of epistasis in biological systems is often unknown.
  • Efficiency through Policy Gradient: Using a policy gradient method allows the agent to learn a strategy for selecting promising SNP combinations without exhaustively searching all possibilities, leading to computational efficiency.
  • Leveraging Existing Metrics: The integration of established epistasis measurement metrics (MDR CCR, Rule Utility) into the RL reward function ensures that the learned policy is guided by biologically relevant signals.

Key Results

In a preliminary study on a simulated 2-locus dataset with 100 SNPs, the EpiRL agent successfully identified the interacting SNPs in 34 out of 50 trials within 5000 iterations. For successful trials, the average number of iterations to find the interaction was 2260.6, and the average time was 22.4 seconds. This is notably faster than an exhaustive search, which took 51 seconds for the same task.

Limitations and Future Work

The authors acknowledge that their preliminary study was conducted on a relatively small dataset (100 SNPs). Future work will involve:

  • Experimenting on larger datasets with various locus interactions.
  • Comparing EpiRL's performance (recall and running time) against existing methods like MDR, BEAM, and Ant Optimization.
  • Applying EpiRL to real-world GWAS datasets, such as the Coronary Artery Disease (CAD) dataset, to validate its effectiveness in a clinical context.

Summary updated

Abstract

Epistasis (gene-gene interaction) is crucial to predicting genetic disease. Our work tackles the computational challenges faced by previous works in epistasis detection by modeling it as a one-step Markov Decision Process where the state is genome data, the actions are the interacted genes, and the reward is an interaction measurement for the selected actions. A reinforcement learning agent using policy gradient method then learns to discover a set of highly interacted genes. Our preliminary study shows a positive result.
BibTeX
@inproceedings{huang2020epirl,
  author = {Kexin Huang and Rodrigo Frassetto Nogueira},
  title = {{EpiRL: A Reinforcement Learning Agent to Facilitate Epistasis Detection}},
  booktitle = {Precision Health and Medicine},
  year = {2020},
  url = {https://openreview.net/forum?id=g2-WpXRxdK-},
}
Figure