TL;DR
The relate framework introduces a two-stage approach to solve the "missing variable" problem in multi-cohort studies. By combining Random Forest-based clustering with recursive statistical testing, it identifies which groups of participants are similar enough to share data, allowing researchers to accurately fill in missing information without introducing the biases of "one-size-fits-all" imputation.
What Problem Does This Paper Address?
In modern epidemiology, researchers often perform "integrative analysis"—pooling data from dozens of independent studies (cohorts) to increase statistical power. However, these studies rarely collect the exact same variables. One cohort might have detailed data on maternal smoking but lack neighborhood-level pollution data, while another has the opposite.
This creates cohort-level missingness. Traditionally, researchers either throw out any cohort missing a variable (losing massive amounts of data) or "impute" the missing values by assuming every cohort in the pool follows the same underlying distribution. The latter is dangerous: if you use data from a high-income urban cohort to fill in gaps for a low-income rural cohort, your final results will be biased. The challenge is identifying which cohorts are actually "distributionally equivalent" so that data can be shared safely.
Methods
The authors propose a method called relate (REcursive muLtivariAte TEsting for cohort clustering), which operates in two distinct stages:
-
Stage 1: Identifying Candidate Groups (The Random Forest Trick) Instead of using a standard distance metric, the authors use a Random Forest (RF) classifier to predict which cohort an individual belongs to based on their covariates. If the RF can easily tell participants apart by cohort, the cohorts are different. If the RF struggles to classify them, the cohorts are likely similar. They then use the "dissimilarity" scores from the RF to build a hierarchical clustering dendrogram (a tree-like diagram of similarity).
-
Stage 2: Formal Recursive Testing To move from "looks similar" to "is statistically similar," the authors apply the Biswas and Ghosh (BG) test, a non-parametric test designed for high-dimensional data. Rather than testing every possible pair of cohorts (which is computationally expensive), they move up the dendrogram created in Stage 1, recursively testing and merging groups only if the null hypothesis of "same distribution" isn't rejected.
Datasets
The paper evaluates the method using both synthetic and real-world data:
- Simulated Data: 10 cohorts with 15 covariates (a mix of normal, binary, binomial, and multinomial distributions) across five different scenarios ranging from "all cohorts are the same" to "four distinct underlying distributions."
- ECHO (Environmental influences on Child Health Outcomes):
- Study 1: Neighborhood-level exposures and pre-term birth (35 cohorts).
- Study 2: Cardiometabolic pregnancy complications and autism spectrum disorder (ASD) traits (28 cohorts).
Key Insights and Ideas
- Classification as a Similarity Metric: The use of a classifier's inability to distinguish between groups as a proxy for distributional similarity is a clever application of machine learning. It bypasses the need for complex parametric assumptions.
- Efficiency through Recursion: Brute-force pairwise testing for 69 cohorts would require over 2,300 tests. By following the hierarchical tree,
relatedrastically reduces the number of tests and the computational time. - Design vs. Reality: A major "aha moment" in the real-world application was that cohorts with similar study designs (e.g., all "autism-enriched" cohorts) did not always have similar data distributions. This proves that manual grouping based on metadata is insufficient; you need a data-driven approach.
Key Results
- High Accuracy: In simulations,
relatecorrectly identified the number of distribution clusters with >90% accuracy in most scenarios, even with high levels of missing data (up to 60%). - Speed: The entire process (RF fitting + recursive testing) typically runs in under two minutes for a 10-cohort setup, making it highly scalable for large consortia.
- Impact on Inference: In the ECHO pre-term birth study, the authors showed that "pooled" imputation (treating all 35 cohorts as one) produced statistically significant results for certain race and ethnicity categories that vanished when using the more precise
relategroupings. This suggests that standard imputation methods may be generating "false positive" associations by ignoring cohort heterogeneity.
Limitations and Future Work
- The MAR Assumption: The method assumes data is "Missing At Random" (MAR), meaning the reason data is missing is related to observed variables, not the missing values themselves.
- Singleton Cohorts: Sometimes a cohort is so unique that it doesn't group with any others. In these cases,
relatecannot perform imputation, and the cohort must be excluded—a "fail-safe" that protects data integrity but reduces sample size. - Small Study Limitations: The authors suggest the method may not be suitable for very small studies (3-4 cohorts) where the statistical tests lack the power to distinguish distributions.
- Future Directions: The authors suggest exploring "superlearning" (combining multiple ML algorithms) to further improve the initial clustering stage.
Summary updated