Separating Batch Distortion from Cell Mixture in DNA Methylation by Using CpG Density

DNA methylation is a chemical mark on DNA that differs by cell type. Methylation profiles from blood, tumors, or cell-free DNA are therefore used to estimate which cells a sample contains. Two problems corrupt these estimates at the same time. First, laboratory "batch effects" (different kits, sequencing centers, or platforms) distort the measured methylation. Second, bulk samples average over many cells, so a change in cell composition can look like a change in methylation. Standard batch correction cannot tell these two apart. If one cohort's cases were processed in a different batch than its controls, correction erases real composition differences along with the artifacts.

This proposal uses a third, usually ignored variable to pull the two apart: local CpG density. A CpG is a cytosine followed by a guanine, the site where methylation sits, and the genome contains dense CpG clusters as well as sparse stretches. Many technical distortions in methylation assays are functions of CpG density. Affinity capture binds dense, methylated regions more strongly, and arrays and capture panels systematically underestimate changes in sparse regions (Liang et al.). Cell-type signals, by contrast, are specific to individual regions and concentrate in sparse distal enhancers (Aran et al.). If each batch's distortion is modeled as a smooth curve indexed by CpG density and shared across all regions of similar density, then what the curve cannot explain must come from the region-specific cell mixture. That structural difference is the handle this proposal relies on.

Core Claim

We claim that one generative model can estimate cell proportions and cell-type methylation profiles from multi-batch, multi-platform bulk methylation data more accurately than "correct batches, then deconvolve" pipelines. The model has three parts: a count likelihood suited to fractional methylation, a per-batch distortion curve over CpG density, and a reference-based mixture of cell types. The gains should appear specifically where the two pipelines disagree, namely when batch is confounded with composition and in low-density enhancer regions. The new ingredient is the density-indexed batch term. It turns batch correction from "make the distributions overlap" into "fit a low-dimensional technical curve anchored on regions that should not change," so composition differences are never targets for removal.

For region $r$ in sample $s$ from batch $b$, with $n_{sr}$ reads covering it and $m_{sr}$ of them methylated, the model is

$$m_{sr} \sim \mathrm{BetaBinomial}\big(n_{sr},\ \mu_{sr},\ \phi_b(d_r)\big), \qquad \mu_{sr} = g_b\Big(d_r,\ \textstyle\sum_k \pi_{sk}\,\theta_{kr}\Big).$$

Here $d_r$ is the region's CpG density, $\pi_{sk}$ is the proportion of cell type $k$ in sample $s$, and $\theta_{kr}$ is the methylation level of cell type $k$ at region $r$ (initialized from a reference atlas and allowed to deviate under a shrinkage prior). The function $g_b$ is a batch-specific distortion that is monotone in the true level and varies smoothly with density. We use a spline surface with roughly ten density knots, with one batch fixed to the identity. The term $\phi_b(d_r)$ is a density- and batch-dependent overdispersion, which lets sparse regions count as less informative rather than being thrown away.

Relation to Prior Work

Three lines of work motivate or constrain this design. scMET supplies the likelihood. It fits Beta-Binomial generalized linear models (regressions with a Beta-Binomial noise model) with CpG density as a covariate, so that single-cell variability is not an artifact of sequence composition. It works within a single dataset, however, has no cross-batch term, and does not deconvolve bulk samples. We reuse its density-aware Beta-Binomial idea but move density from a nuisance covariate into the structure of the batch effect.

cfDecon is the closest existing method and our main competitor. It deconvolves cell-free DNA using read-level methylation histograms in fixed 500 bp windows, with an autoencoder that outputs proportions and condition-aware signatures. It treats every window as equally reliable regardless of CpG count, and it does not separate sequencing-center effects from disease signal. Our model differs in both respects.

Long-read enhancer–promoter deconvolution motivates the focus on sparse regions: adding enhancers to promoters lowered mixture error. It was tested only on cell lines processed under uniform conditions.

The batch-correction literature constrains the design. BERMUDA showed that global distribution matching collapses cell types when batch compositions are unbalanced. Its fix, aligning only similar clusters, depends on a threshold that is hard to set without labels. scAdapt and latent-space methods such as scBatchProx align embeddings adversarially or post hoc. All of these assume count or continuous data and single cells. None models bounded methylation fractions or density-dependent distortion. Cross-platform microarray work (Warnat et al.) shows that global normalization leaves residual lab clustering. We do not align distributions at all. Composition enters only through $\pi$, so it can never be "corrected away."

Research Program

Identifiability comes from anchors. Aran et al. describe promoters that stay unmethylated in every cell type ("constantly open"), and many open-sea regions stay methylated in every cell type. Anchor regions are chosen from the reference atlas as those with low variance across all cell types, and they are spread across density bins. Across batches they should differ only through $g_b$. They therefore pin down the distortion curve at every density before any cell-type-variable region is used. Fitting alternates between two steps. The first updates proportions and signature deviations given the curves, using an expectation-maximization step on the Beta-Binomial model. The second updates the curves given the proportions, using penalized spline regression. Because $g_b$ has about a few dozen parameters per batch against hundreds of thousands of regions, it cannot absorb region-specific cell-type structure. We will test that claim directly rather than assume it.

We will build the model, a simulator of density-dependent batch distortion, and an analysis of when the density axis actually carries the separating information. The simulator will have three distortion types: affinity-capture enrichment increasing with density, coverage dropout that is worse in sparse regions, and platform compression of intermediate values.

Baselines

The naive reference point is reference-based deconvolution on raw pooled data with no batch handling, using non-negative least squares against the atlas. It sets the floor and shows how much damage batch distortion does.

The strongest standard pipeline is location-scale batch correction (ComBat-style adjustment on logit-transformed methylation fractions) with batch as the only covariate, followed by the same deconvolution. A second variant adds the case/control label as a protected covariate, which is how practitioners try to avoid over-correction. The protected variant is the fairest version of the "correct then deconvolve" approach.

The strongest existing deconvolver is cfDecon. It will be run both on raw pooled data and after ComBat correction, with the same reference atlas, the same regions, and the same number of hyperparameter trials as our model. The file reports that cfDecon achieves higher concordance correlation and lower L1 error than CelFEER on ENCODE/UXM simulated mixtures and on the ALS and HCC cohorts. However, no reported numerical result was found on Lacuna for cfDecon's concordance or L1 values. We will therefore reproduce it ourselves rather than quote a number. For context on sparse-region value, the long-read study reports a mean mixture error of 0.69% using enhancer–promoter pairs with maximum-likelihood estimation versus 0.86% for promoters only. That figure comes from a different, single-batch assay and is not a direct target.

Two ablations isolate the key ingredient. The first replaces $g_b(d_r,\cdot)$ with a density-independent per-batch monotone curve, which is joint deconvolution with a global batch correction. The second keeps the density curve but fixes $\phi$ constant, which removes the density-aware weighting. Beating a baseline means lower L1 proportion error and higher concordance, with paired bootstrap intervals over mixtures that exclude zero. It also means, under confounded designs, recovering the true case-versus-control composition difference that the correction pipelines shrink.

Why not just…?

Why not just run ComBat, protecting the phenotype label, and then deconvolve? Protecting the label keeps label-associated variation, but composition shifts are not the only label-associated signal, and the protected variant still assumes the batch shift is the same at every region. If distortion depends on density, a single location-scale shift under-corrects sparse enhancers and over-corrects dense islands. That leaves exactly the region-dependent error that deconvolution misreads as composition. Real batches are also often perfectly confounded with label, and then protection becomes unidentifiable. The confounded-design experiment below settles this directly.

Why not just add CpG density as a covariate, as scMET does? A covariate adjusts the mean for density but does not let density modulate the batch effect. It would need a batch-by-density interaction, which is our model under another name, but without anchors and without coupling to the mixture. The ablation that removes density from $g_b$ measures how much this structure contributes. If the density-independent curve performs as well, the density claim is wrong and we will report joint deconvolution with batch calibration as the finding.

Why not deconvolve each batch separately, since proportions are within-sample quantities? This is the most serious alternative. Within-batch deconvolution avoids cross-batch alignment entirely. It still compares every batch against an atlas made on another platform, so a density-dependent mismatch biases proportions in a batch-specific way, and the bias shows up as spurious between-batch composition differences. Per-batch deconvolution will be run as an additional baseline. If distortion in real platform pairs turns out to be mild, this simple method may match ours, and we will say so.

Why not restrict to CpG islands, where arrays and capture are reliable? Islands are often unmethylated in all cell types, and cell-type information concentrates in sparse enhancers (Aran et al.; Trump et al., who found that 95% of stress-associated regions would have been missed by standard arrays). An island-only deconvolution run will quantify what is lost.

Proposed Experiments

The first experiment is semi-synthetic. We will mix cell-type profiles from the ENCODE and UXM 35-cell-type whole-genome bisulfite atlas used by cfDecon at known proportions, including rare types at 1–5%. We will assign mixtures to three to six batches and apply each simulated distortion type at graded strengths. The decisive design is confounded: cases, enriched for one cell type, sit mostly in one batch. We report proportion L1 error, concordance correlation, and the estimated case–control difference in the enriched type relative to its true value. Our model should keep most of that difference, while the ComBat pipelines should shrink it.

The second experiment tests the identifiability claim. We plant deliberately density-correlated cell-type signal, meaning a cell type whose marker regions are all sparse, and check whether $g_b$ wrongly absorbs it. This is the setting where the method should fail, and we will measure how badly.

The third experiment uses real distortion. We will estimate density-dependent distortion curves between whole-genome bisulfite and array- or capture-based measurements of the same atlas cell types where both exist, then fit and apply them to mixtures. The purpose is to test whether real platform differences actually vary with density, as Liang et al. suggest. If the fitted curves are flat, the core premise fails.

The fourth experiment uses the ALS and HCC cell-free DNA cohorts analyzed by cfDecon, treating sequencing run or center as batch where available. Without ground truth, the checks are internal. Estimated proportions should not separate by batch within the same diagnosis group, while known biology should persist, such as the elevated liver-derived fraction expected in HCC.

Across all experiments, we will separately report the recovered cell-type signature deviations in sparse versus dense regions. This tests whether density-aware modeling specifically helps recover the enhancer-level, cell-type-specific signal that bulk averaging hides.