Abstract
Randomized experiments (REs) are the cornerstone for treatment effect evaluation. However, due to practical considerations, REs may encounter difficulty recruiting sufficient patients. External controls (ECs) can supplement REs to boost estimation efficiency. Yet, there may be incomparability between ECs and concurrent controls (CCs), resulting in misleading treatment effect evaluation. We introduce a novel bias function to measure the difference in the outcome mean functions between ECs and CCs. We show that the ANCOVA model augmented by the bias function for ECs renders a consistent estimator of the average treatment effect, regardless of whether or not the ANCOVA model is correct. To accommodate possibly different structures of the ANCOVA model and the bias function, we propose a double penalty integration estimator (DPIE) with different penalization terms for the two functions. With an appropriate choice of penalty parameters, our DPIE ensures consistency, oracle property, and asymptotic normality even in the presence of model misspecification. DPIE is at least as efficient as the estimator derived from REs alone, validated through theoretical and experimental results.
Introduction
Randomized experiments (REs), such as clinical trials or A/B tests, are the gold standard for determining treatment efficacy. However, they are often "sample hungry"—recruiting enough participants is expensive and time-consuming. To address this, researchers often leverage External Controls (ECs)—data from previous studies or electronic health records.
The primary challenge is that external data is often "incomparable" to the current trial due to unmeasured differences in patient populations or environments, introducing bias. Standard methods often rely on the "exchangeability" assumption, which is rarely met in practice. If the underlying model is misspecified, integrating external data can lead to erroneous conclusions.
Methodology: The Double Penalty Integration Estimator (DPIE)
The proposed DPIE is a model-robust approach that integrates REs and ECs without requiring perfectly specified models. It is built on three pillars:
- The Bias Function: A function $b_0(X)$ is introduced to explicitly measure the discrepancy between external data and the concurrent control group. Adding this to a standard ANCOVA model "neutralizes" inherent bias.
- Sieve Estimation: To handle unknown shapes of bias and outcome functions, the method uses "sieves" (basis functions like polynomials) to flexibly approximate complex relationships.
- Double Penalty: In high-dimensional settings, the outcome model and the bias function likely have different levels of complexity. DPIE applies two distinct penalty parameters ($\lambda_1$ and $\lambda_2$) for these functions. This allows the estimator to achieve the "oracle property," performing as well as if the important variables were known a priori.
Experimental Results
The DPIE method was validated using synthetic simulations and the National Supported Work (NSW) study benchmark.
Simulation Performance
Simulations demonstrated that DPIE consistently achieves lower Mean Squared Error (MSE) compared to using only RE data or a single-penalty approach (SPIE). As shown in Figure 1, DPIE maintains a low and stable MSE even as the ratio of bias increases, whereas SPIE's error grows significantly.
Real-World Application: NSW Study
Applying DPIE to the NSW job training data—where the Current Population Survey (CPS) served as the external control—demonstrated a clear boost in statistical power. DPIE produced a standard error of 0.567, significantly lower than the 0.634 achieved using only experimental data.
Discussion and Future Work
DPIE offers a significant safety net for practitioners by ensuring consistency even under model misspecification. The integration of external data via this method is proven to be at least as efficient as using randomized experiments alone.
However, the method involves higher computational complexity due to the search for two optimal tuning parameters. Future research will focus on extending this framework to more complex data types, such as survival analysis and zero-inflated outcomes, and addressing the challenges of non-convex optimization in high-dimensional spaces.
Summary updated