Detecting statistical interactions in immune receptor data: a comparative study

Thomas Minotto, Ingrid Hobæk Haff, Enrico Riccardi, Geir Kjetil SandveJournal of Applied Statistics 2025 · DOI

TL;DR

This paper compares various machine learning methods for detecting statistical interactions in immune receptor data, crucial for understanding how these receptors bind to antigens. They found that while pairwise interactions are readily detected, higher-order interactions are more challenging, with neural networks offering a speed advantage but tree-based methods showing better performance for complex interactions.

What Problem Does This Paper Address?

Understanding how immune receptors, like antibodies, bind to antigens is fundamental to immunology and drug development. This binding process is complex, involving intricate interactions between amino acids within the receptor and the antigen. While advanced machine learning models can predict binding, they often act as "black boxes," making it difficult to pinpoint which specific amino acid interactions are most important. This lack of interpretability hinders our ability to design better therapeutics or understand disease mechanisms. The paper aims to address this by comparing different statistical and machine learning methods specifically for detecting these crucial interaction effects in immune receptor data.

Methods

The authors evaluated six different interaction detection methods:

  • Logistic Lasso-based methods (glinternet, hierNet): These methods extend logistic regression by using regularization (lasso) to select important main effects and pairwise interactions, with constraints to enforce a hierarchy (i.e., interactions are only considered if their constituent main effects are also important).
  • Logic Regression-based methods (MCLR, logicFS, BLR): These methods use Boolean logic trees to model interactions. They explore combinations of features (amino acids) to identify interaction patterns.
  • Random Forest-based method (iRF): This approach uses an iterative random forest algorithm to extract interactions by analyzing how often feature combinations appear in decision trees.
  • Neural Network-based method (NID): This method fits a neural network and then analyzes the network's weights to infer interaction strengths.

The study compared these methods based on their ability to detect interactions, the impact of various factors (interaction strength, order, frequency, data size), and their computational efficiency.

Datasets

The study utilized two types of datasets:

  • Simulated Immune Receptor Data: Generated using the OLGA software, which mimics the V(D)J recombination process. This allowed for controlled simulation of interactions with known properties (strength, order, implantation rate) to benchmark the methods. Both discriminative and generative models were used for simulation.
  • Experimental Immune Receptor Data: A real-world dataset of antibody sequences binding to the HER2 antigen, containing 45,361 sequences. This dataset was used to evaluate the methods' performance on practical, less controlled data.

Key Insights and Ideas

  • Pairwise interactions are surprisingly detectable: Even with just 1000 sequences, most methods could reliably detect simple pairwise interactions. This suggests that significant insights can be gained from relatively modest datasets.
  • Higher-order interactions are a real challenge: While pairwise interactions were often found, detecting interactions involving three or more amino acids proved much harder. These higher-order interactions were frequently confused with their simpler sub-interactions.
  • Interaction strength matters most: The impact of an interaction on the binding outcome needs to be substantial, ideally comparable to or greater than the main effects, for detection methods to succeed.
  • Implantation rate is a sweet spot: Surprisingly, the best detection performance wasn't always at the highest implantation rate (50%). Rates around 10-20% often yielded optimal results, suggesting a balance between signal strength and avoiding overfitting or masking by noise.
  • Neural networks are fast, but not always best for complexity: The neural network approach (NID) was significantly faster than others, especially for large datasets. However, for detecting complex, higher-order interactions, tree-based methods (logic regression and random forests) generally performed better.
  • Hierarchy assumptions don't always help: Methods that enforce a hierarchy (interactions only if main effects are present) didn't necessarily perform better, and sometimes performed slightly worse, than those without such assumptions.

Key Results

  • Two-way interactions: Detected with high accuracy (over 80% of datasets) by most methods, even with only 1000 sequences.
  • Three-way interactions: Detection performance dropped significantly. Only MCLR could reliably detect three-way interactions across many datasets, though performance improved with larger sample sizes (10,000 sequences).
  • Computational Efficiency: NID was orders of magnitude faster than other methods. glinternet and NID could handle the largest datasets (100,000 sequences) within a reasonable time. MCLR was computationally intensive, taking about 60 hours for 10,000 sequences.
  • Experimental Data: The study identified 14 pairwise interactions and one three-way interaction in the HER2 binding data. Including these interactions in a logistic regression model led to a minor improvement in AUC and average precision, suggesting they provide valuable, albeit subtle, insights.

Limitations and Future Work

  • Higher-order interaction detection: The paper highlights that detecting and correctly ranking higher-order interactions remains a significant challenge. Methods often detect sub-interactions instead.
  • Computational Cost: While NID is fast, other methods, particularly for higher-order interactions, are computationally demanding, limiting their scalability to extremely large datasets.
  • Data Requirements: Detecting interactions requires a sufficient fraction of sequences to contain the interaction and enough data overall (thousands of sequences).
  • Future Work: The authors suggest that neural network-based models hold significant potential for future development in interaction detection methods due to their speed and ability to handle complex patterns. Further research could focus on improving the interpretability of these models for higher-order interactions. The findings are also applicable beyond immune receptors to other biological binding processes.

Summary updated

Abstract

Statistical interactions are part of numerous data generating processes and several methods have been developed to detect them. We here study immune receptors binding to antigens, where advanced machine learning techniques have proved useful for binding prediction, suggesting significant intra amino acid chain interactions. We reviewed detection methods based on logistic lasso, logic regression, random forests and neural networks. We compared detection performance in simulated immune data, and how it is affected by the order of interactions, their strength related to the main effects, their frequency of occurrence and the size of the data. Interactions were implanted as motifs of amino acids that determined the binding status of sequences through a logistic regression model. Results show that pairwise interactions were retrieved from just 1000 sequences in the dataset, and optimal detection happened for an implantation rate of around 20 percent. For higher-order interactions, the best performance was obtained by logic regression and random forest based methods. The running time for the neural network-based method was several orders of magnitude lower, followed by the lasso-based methods. We applied the methods on an experimental dataset and identified several pairwise interactions as well as a three-way interaction, enhancing the accuracy of prediction models.
BibTeX
@article{minotto2025detecting,
  author = {Thomas Minotto and Ingrid Hobæk Haff and Enrico Riccardi and Geir Kjetil Sandve},
  title = {{Detecting statistical interactions in immune receptor data: a comparative study}},
  journal = {Journal of Applied Statistics},
  year = {2025},
  doi = {10.1080/02664763.2025.2533483},
}

Loading related content...

Figure