Sample Size Dependent Estimator Selection

19 concepts11 papers1 proposal

When evaluating a new policy, treatment, or decision-making strategy using historical data, statisticians and machine learning practitioners inevitably run into the bias-variance tradeoff.

On one hand, we have unbiased estimators like Importance Sampling (IS) or Inverse Probability Weighting (IPW). These methods provide strong theoretical guarantees because they do not rely on assumptions about the environment's underlying dynamics. However, if the historical behavior policy differs significantly from our target policy, the importance weights explode, leading to catastrophic variance. On the other hand, we have model-based estimators that predict rewards directly. These estimators are highly stable (low variance) but suffer from systematic bias if our predictive model is misspecified.

Historically, practitioners treated the choice between these estimators as a static architectural decision. However, a growing body of research demonstrates that the optimal point on the bias-variance frontier is fundamentally a function of sample size and data quality. By dynamically shifting estimation strategies based on the volume of available data, we can achieve significantly lower Mean Squared Error (MSE) than any single universal estimator can provide.

The Intuition: Why the Optimal Estimator Shifts

To understand why our choice of estimator must change with sample size, consider how the components of Mean Squared Error behave as data accumulates:

$$\text{MSE} = \text{Variance} + \text{Bias}^2$$

The variance of an estimator is highly sensitive to the sample size $N$, typically scaling at a rate of $\mathcal{O}(1/N)$. In contrast, the bias introduced by an imperfect reward model or a regularized parameter is often a fixed constant determined by model misspecification.

Small Data Regime (Low N)          Large Data Regime (High N)
+---------------------------+      +---------------------------+
| Variance is dominant      |      | Variance naturally shrinks|
| Error ≈ Variance          |      | Error ≈ Bias^2            |
| Strategy: Suppress var    |      | Strategy: Eliminate bias  |
| (Favor biased models)     |      | (Favor unbiased sampling) |
+---------------------------+      +---------------------------+

When data is scarce (small $N$), the variance term completely dominates the total error. A single extreme importance weight can ruin an entire evaluation. In this regime, the most effective strategy is to aggressively suppress variance, even if it means accepting a heavily biased estimate.

As data volume scales (large $N$), the variance term naturally shrinks toward zero. At this point, the constant bias of our predictive models becomes the primary bottleneck. In this large-data regime, we should shed our variance-reducing constraints and transition toward unbiased, sample-driven estimators.

This sample-size dependency has been validated across multiple domains:

  • Off-Policy Benchmarks: Using the Open Bandit Dataset, Saito et al. demonstrated in A Large-scale Open Dataset for Bandit Algorithms and Evaluating the Robustness of Off-Policy Evaluation that simple direct reward models provide superior robustness when data volume is low, whereas importance-weighting methods dominate as data scales because their bias diminishes.
  • Offline Reinforcement Learning: Hu et al. showed in On the Role of Discount Factor in Offline Reinforcement Learning that the discount factor ($\gamma$) acts as a variance-reducing regularizer. Smaller or noisier datasets require a lower discount factor to maintain stability, while the optimal $\gamma$ shifts toward the true evaluation discount as data volume increases.
  • Regression and Optimization: Yi-Hao Kao proved in Directed Regression that combining general predictive models with task-specific empirical optimization yields a hybrid algorithm where the optimal balance shifts toward aggressive task-specific tuning only as data availability grows.

Mechanisms for Smooth Interpolation

Rather than making a binary choice between a biased model and an unbiased sample, modern frameworks introduce continuous hyperparameters that allow an estimator to smoothly slide along the bias-variance frontier.

Weight Flattening and Trimming

In causal inference and off-policy evaluation, extreme propensity scores create heavy-tailed weight distributions. To counter this, Hachiya et al. in Adaptive Importance Sampling with Automatic Model Selection in Value Function Approximation utilize a "flattening parameter" to suppress importance weights in data-sparse regimes. Similarly, Metelli et al. developed an interpolation parameter in Subgaussian and Differentiable Importance Sampling for Off-Policy Evaluation and Learning that stabilizes importance sampling based on sample-based estimates of policy divergence.

Threshold-Based Switching

Instead of blending weights, the SWITCH estimator introduced by Wang et al. in Optimal and Adaptive Off-policy Evaluation in Contextual Bandits applies a hard threshold to individual data points. For actions where the importance weight is below a certain threshold (low variance), it uses the unbiased sample weight. For actions where the importance weight explodes (high variance), it drops back to a biased reward model.

SWITCH Estimator Logic:
Given weight W and Threshold τ:
If W ≤ τ  ==> Use Unbiased Importance Weight (Low Variance Region)
If W > τ  ==> Use Biased Reward Model Model   (High Variance Region)

Automated Selection: Approaching the Oracle

The ultimate goal of this research direction is to automate hyperparameter tuning and estimator selection without requiring prior knowledge of the underlying data distributions. How can an algorithm find the optimal threshold purely from the observed data?

Lepski’s Principle (The SLOPE Method)

In Adaptive Estimator Selection for Off-Policy Evaluation, Su et al. introduced SLOPE, a data-driven method that leverages Lepski’s principle from nonparametric statistics.

The core idea is to construct a sequence of estimators ordered from highest bias/lowest variance to lowest bias/highest variance. The algorithm steps through this sequence, comparing each new estimator against all preceding ones. It stops as soon as the difference between a high-variance estimate and a lower-variance baseline exceeds a threshold proportional to the baseline's standard deviation. This allows the system to automatically navigate the Pareto frontier and perform near the level of an omniscient oracle.

Empirical MSE Minimization

Another prominent strategy is to directly construct and minimize an empirical proxy of the total MSE objective.

By treating the bias-variance tradeoff as a dynamic, sample-size dependent optimization problem, these methods allow data systems to remain highly stable when data is scarce, while unlocking maximum precision as data scales.

Key Foundational Work

Adaptive Importance Sampling with Automatic Model Selection in Value Function Approximation (2008) — contributed 2 concept elements to this direction.
Directed Regression (2009) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_sample_size_dependent_estimator_selectio_65602c1b,
  title = {sample-size dependent estimator selection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sample-size-dependent-estimator-selection/rdr_65602c1bd2964890a02e6110fd4c0fc5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_sample_size_dependent_estimator_selectio_65602c1b_v1,
  title = {sample-size dependent estimator selection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sample-size-dependent-estimator-selection/rdr_65602c1bd2964890a02e6110fd4c0fc5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake