TL;DR
This paper introduces a novel random forest (RF) based methodology for analyzing high-dimensional longitudinal data, addressing limitations of existing methods. The proposed approach, called SREEMforest, extends random forests to handle complex covariance structures over time and accounts for individual-specific correlations, demonstrating superior performance in simulations and a real-world HIV vaccine trial.
What Problem Does This Paper Address?
Analyzing data where observations are repeatedly measured over time (longitudinal data) is common in many scientific fields. When this data is also "high-dimensional," meaning there are many more predictor variables (e.g., gene transcripts) than observations (e.g., patients), standard statistical methods struggle. Random forests (RFs) are powerful machine learning tools that perform well in high-dimensional settings, but adapting them to longitudinal data while accounting for the complex correlations within individuals has been a challenge. Existing methods often simplify the covariance structure or don't fully leverage the information from repeated measurements. This paper aims to bridge this gap by developing an RF-based approach that can effectively handle both high dimensionality and the intricacies of longitudinal data.
Methods
The core of this paper is the development of a stochastic semi-parametric mixed-effects model for longitudinal data. This model allows for:
- A flexible mean behavior function: This part is estimated using random forests, allowing for complex, non-linear relationships.
- Random effects: These capture individual-specific deviations from the average trend.
- A stochastic process: This models the serial correlation within an individual's measurements, allowing the covariance structure to vary over time.
The authors propose SREEMforest, an extension of random forests that incorporates this stochastic mixed-effects model. Key innovations include:
- Intra-individual covariance consideration: The forest building process explicitly accounts for the correlation structure within each individual, leading to more robust trees.
- Stochastic extension: The model allows for time-varying covariance, making it more adaptable to real-world data where correlations might change.
- EM Algorithm: An Expectation-Maximization algorithm is used to iteratively estimate the model parameters, including the random effects and the mean behavior function.
They also introduce related methods like MERF (non-stochastic version) and REEMforest/SREEMforest (which use CART trees or RFs, respectively, to model the mean behavior function and incorporate intra-individual covariance).
Datasets
The authors evaluate their proposed methods using:
- Simulated datasets: They generate data under two models (non-stochastic and stochastic) with varying dimensions (low-dimensional with p=6 and high-dimensional with p=8000) to assess performance.
- DALIA vaccine trial data: This real-world dataset consists of longitudinal measurements from 17 HIV-infected patients, including 20,000 gene transcripts and HIV viral load data. This dataset is used to demonstrate the practical application and effectiveness of their approach.
Key Insights and Ideas
- The power of accounting for intra-individual correlation: Explicitly modeling the covariance structure within individuals significantly improves the estimation of the mean behavior function and reduces prediction error, especially when using random forests.
- Stochasticity matters: Allowing the covariance structure to vary over time (stochastic process) further enhances the model's ability to capture complex longitudinal dynamics.
- Random forests are well-suited for high-dimensional longitudinal data: The proposed SREEMforest method demonstrates superior performance compared to existing tree-based and linear mixed-effects models in both simulated and real-world scenarios.
- The importance of the
mtryparameter: For iterative RF-based algorithms like the EM algorithm used here, a sufficiently largemtry(number of variables randomly sampled at each split) is crucial for stabilizing the algorithm and ensuring convergence.
Key Results
- Simulation Studies:
- In low-dimensional settings, SREEMforest consistently showed lower squared biases for parameter estimations compared to other methods, including linear mixed models and tree-based approaches.
- In high-dimensional settings, SREEMforest and MERF (its non-stochastic counterpart) achieved significantly lower prediction errors than tree-based methods and even Breiman's standard RF, which doesn't account for longitudinal structure.
- The proposed forest-based methods (SMERF and SREEMforest) generally outperformed their tree-based counterparts (SMERT and SREEMtree) and linear mixed models in terms of bias and prediction error.
- DALIA Vaccine Trial Application:
- The SREEMforest method successfully identified 21 gene transcripts associated with HIV viral load dynamics.
- These selected transcripts were largely linked to the interferon-alpha pathway, which is known to be involved in the immune response to viral infections, providing biological validation for the method.
- The method demonstrated a significant drop in prediction error by effectively incorporating the longitudinal aspect of the data.
Limitations and Future Work
- Theoretical Analysis: The authors acknowledge that the theoretical analysis of complex methods involving non-parametric estimates within EM algorithms is challenging and remains an open issue.
- Split Criterion: They suggest that adapting the split criterion in tree building, similar to how it's done for survival RFs, could be a future direction for improving longitudinal RF methods.
- Random Effects Structure: While the current application uses a random intercept, the authors note that in contexts with more individuals or fewer time points, adding random effects for different time points could enhance model flexibility and better capture inter-individual variability.
mtryParameter Optimization: While they provide guidance on choosingmtry, systematic optimization for this parameter could be explored further.
Summary updated