TL;DR
The iterative Random Forest (iRF) algorithm provides a computationally efficient way to discover high-order feature interactions (e.g., 5-way or 6-way combinations) in complex datasets. By combining iterative feature weighting with Random Intersection Trees and stability testing, iRF identifies biologically meaningful "molecular recipes" without the exponential search costs typically associated with high-order interaction discovery.
What Problem Does This Paper Address?
In genomics, biological processes are rarely driven by a single protein or gene acting in isolation. Instead, they are governed by "molecular machines"—complex assemblies where multiple transcription factors (TFs) and histone modifications interact.
Statistically, finding these interactions is a nightmare. Most traditional methods focus on pairwise relationships (A interacts with B). To find a 5th-order interaction (A, B, C, D, and E acting together), a brute-force search would have to evaluate an astronomical number of combinations, leading to a "combinatorial explosion." Furthermore, genomic data is notoriously noisy, making it difficult to distinguish true biological interactions from random correlations. The authors sought a method that is both computationally scalable and statistically "stable" (reproducible).
Methods
The iRF algorithm consists of three main stages:
- Iterative Feature Weighting: The algorithm starts by training a standard Random Forest (RF). It takes the "Gini importance" (a measure of how useful a feature is for prediction) and uses it to weight the features in the next iteration. This "zooms in" on the most relevant features, effectively performing soft dimension reduction and encouraging the trees to use a consistent set of important features.
- Generalized Random Intersection Trees (RIT): To find interactions, iRF doesn't search all possible combinations. Instead, it looks at the decision paths already built by the RF. It uses the RIT algorithm to identify sets of features that frequently co-occur on paths leading to a specific prediction. This decouples the order of the interaction from the computational cost.
- Stability Bagging: To ensure the discovered interactions aren't just flukes of the specific data sample, iRF uses an "outer layer" of bootstrapping. It runs the entire process multiple times on different subsets of the data and assigns a "stability score" based on how often an interaction is recovered.
Datasets
The authors validated iRF using three distinct approaches:
- Biologically Inspired Simulations: Synthetic data generated using Boolean rules (AND, OR, XOR) to see if iRF could recover known "ground truth" interactions of up to 8th order.
- Drosophila Enhancer Activity: Data from the Berkeley Drosophila Transcriptional Network Project (BDTNP), involving 23 transcription factors and 13 histone modifications across ~7,800 genomic sequences.
- Human Alternative Splicing: Data from the ENCODE consortium for the K562 cell line, involving 253 ChIP-seq assays (107 TFs and 11 histone marks) to predict exon splicing.
Key Insights and Ideas
- The Stability Principle: A core philosophy of the paper (and the Yu group) is that for a result to be scientifically meaningful, it must be stable to small perturbations in the data. iRF bakes this directly into the algorithm via bagging.
- Regularization via Iteration: Iterative weighting acts as a powerful regularizer. It forces the model to find a "stable" path through the feature space, which is particularly useful in "Big P, Small N" scenarios where there are far more features than observations.
- Efficiency: Unlike previous benchmarks like Rulefit3, which grow exponentially in complexity as you look for higher-order interactions, iRF’s computational cost is roughly the same as a standard Random Forest.
- Biological "AND" Gates: The results suggest that many genomic processes function like complex logic gates. For example, a specific enhancer might only activate if three specific proteins are all present at high levels.
Key Results
- High Precision: In the Drosophila case study, 80% of the pairwise interactions iRF identified as "stable" had been previously documented as physical interactions in the literature.
- Novel Discoveries: iRF identified a stable 3rd-order interaction in Drosophila involving the factors Zelda (Zld), Giant (Gt), and Twist (Twi).
- Complex Splicing Code: In human cells, iRF discovered 5th and 6th-order interactions related to H3K36me3 and RNA Pol II. This suggests a highly complex, multi-valent "chromatin code" for splicing that involves many marks acting in concert.
- Scalability: In runtime tests, iRF's computational cost grew linearly with the number of features, whereas the benchmark Rulefit3 grew exponentially, making iRF much more suitable for modern "omics" datasets.
Limitations and Future Work
- Tree Depth Constraints: The maximum order of interaction iRF can detect is limited by the depth of the trees in the Random Forest. If an interaction involves 10 factors but the trees only have 5 levels, it won't be fully captured.
- Global vs. Local Importance: The current iteration uses a global measure of feature importance. The authors note that moving toward "local" importance (how important a feature is for a specific observation) could help find interactions that only occur in rare cell types or specific conditions.
- Mathematical Form: While iRF identifies which features interact, it doesn't explicitly define the precise mathematical function of that interaction (though the authors use response surfaces to visualize them as AND-like rules).
Summary updated