Bulk RNA sequencing measures the average gene expression of a tissue sample, blending every cell type together. Deconvolution tries to recover what fraction of the sample each cell type contributes, using single-cell RNA sequencing as a reference for what each cell type looks like. Current methods break in two ways that a large breast-cancer benchmark documents. When a tumour sample is more than 70–80% cancer cells, they misread normal epithelial cells as cancer cells. They also cannot detect rare immune and stromal populations without also reporting populations that are not there. This note argues that both failures come from the same restriction: deconvolution has stayed convex where convexity cannot express what is needed. A convex loss must keep listening to genes whose reference profile is simply wrong for this patient. A convex sparsity penalty does nothing at all on cell fractions, which always sum to one.
The proposal is a deconvolution objective with two non-convex parts. The first caps how much any single gene can contribute to the misfit, so genes that disagree with the reference are effectively dropped for that sample. The second charges a fixed cost for every cell type that is included at a small fraction, but no further cost once the fraction is large. Both parts are differences of two convex functions. That structure lets a standard difference-of-convex algorithm, which repeatedly replaces the hard part of the objective with a straight-line upper bound, solve the problem as a sequence of tiny convex quadratic programs with guaranteed descent. The test is whether this fixes the high-purity misclassification and improves the false-positive/false-negative trade-off on the benchmark that exposed both problems.
Core Claim
Let $y \in \mathbb{R}^G$ be a bulk sample's expression over $G$ genes and $A \in \mathbb{R}^{G\times K}$ the single-cell reference of $K$ cell-type profiles. Let $w$ be the cell fractions, constrained to the simplex $\Delta$ (non-negative, summing to one), and $\sigma_g$ a per-gene noise scale. We solve
$$\min_{w\in\Delta}\ \sum_{g=1}^{G}\min\!\Big(\tfrac{(y_g-(Aw)_g)^2}{\sigma_g^2},\,\tau\Big)\;+\;\lambda\sum_{k=1}^{K}\min(w_k,\epsilon).$$
Here $\tau$ caps each gene's contribution, $\epsilon$ is the fraction above which a cell type pays no extra sparsity cost, and $\lambda$ sets the sparsity strength. Both terms are differences of convex functions. The capped loss equals the squared loss minus $\max(r_g^2/\sigma_g^2-\tau,0)$, where $r_g = y_g-(Aw)_g$ is the gene's residual. The capped penalty equals $\sum_k w_k - \sum_k\max(w_k-\epsilon,0)$, and on the simplex $\sum_k w_k = 1$ is a constant.
That last identity is why the sparsity term cannot be convex. On the simplex, the convex $\ell_1$ (Lasso) penalty is identically one and selects nothing. Any penalty that pushes small fractions to exactly zero must therefore be non-convex.
The claim to test has two parts. First, sample-specific trimming of mismatched genes is what resolves the high-purity misclassification. The hypothesis is that a patient's cancer cells express programs absent from the reference, and the resulting residuals get "explained" by inflating lineage-similar normal epithelium. Second, the capped-fraction penalty improves rare-lineage detection at a matched false-positive rate.
A difference-of-convex step is cheap here. Linearizing the subtracted terms at the current $w$ removes the currently trimmed genes from the fit and adds a linear reward for cell types already above $\epsilon$. Each step is therefore a weighted non-negative least squares on a simplex with roughly 10–30 variables. Iterates decrease the objective monotonically and converge to critical points.
We are honest about lineage. The gene-trimming half, solved this way, is iteratively trimmed least squares from robust statistics. What is new is the joint formulation with simplex sparsity, its convergence guarantee, and the targeted test against the documented failure modes of deconvolution.
Relation to Prior Work
The motivating evidence comes from the benchmark of nine deconvolution methods over more than 38,000 breast-cancer pseudo-bulk mixtures (Performance of tumour microenvironment deconvolution methods in breast cancer using single-cell simulated bulk mixtures). That study found the purity trap and the false-positive/false-negative tension, with hspe suppressing false positives at a high false-negative cost. It ranked BayesPrism and DWLS best on granular immune lineages. It diagnoses the failures but proposes no objective that targets them; we do.
SCDC attacks reference mismatch differently. It deconvolves against several single-cell references and averages them with weights chosen to minimise reconstruction error. This handles platform-level bias between whole references. It does not handle gene-level, patient-specific mismatch inside a reference, which is what trimming targets. The two approaches are complementary, and the capped loss can be applied within each SCDC reference.
The method comes from difference-of-convex optimization. The basic concave–convex linearization gives the solver. DC Proximal Newton shows that capped-$\ell_1$-type penalties stay accurate as irrelevant correlated features grow, which motivates the sparsity term. Its second-order machinery is unnecessary at $K\approx 20$, so we do not use it. Distributed Nonconvex Optimization for Sparse Representation studies the same penalty family on networks, but has no simplex constraint and no transcriptomic data.
We deliberately do not pursue the other bridges the file suggests. Difference-of-convex Langevin sampling from DC-LA and constrained difference-of-convex error control from Enforcing Fair Predicted Scores on Intervals of Percentiles by Difference-of-Convex Constraints are both plausible. Neither, however, addresses a mechanism we can name for the purity trap. Uncertainty over a point estimate that is systematically biased is not the first thing to fix.
Research Program
The work has three parts. The first is the solver: a difference-of-convex loop with a small simplex quadratic program inside. It starts from a non-negative least squares solution plus a few random simplex starts, and keeps the lowest objective.
The second is the mechanism study. In simulation we know which genes differ between a held-out patient's cancer cells and the reference cancer profile. We will measure whether the genes the method trims are enriched for those differences, and whether trimming them is what removes the inflated normal-epithelial fraction. If trimmed genes are unrelated to true mismatch, the gene-trimming hypothesis is wrong, even if accuracy improves.
The third is tuning without ground truth. The noise scale $\sigma_g$ comes from pseudo-bulk replicate variance, and $\tau$ from a robust quantile of standardized residuals. We will choose $\lambda$ and $\epsilon$ on held-out pseudo-bulks from reference patients only. Then we report how flat performance is across a grid, because a method that needs per-dataset tuning is not usable clinically.
Baselines
The naive reference point is plain non-negative least squares on the simplex with the same reference matrix and gene set. It is the convex problem our objective reduces to when $\tau\to\infty$ and $\lambda=0$, so any gain is attributable to the two non-convex terms. Beating it on high-purity normal-epithelial error is the minimum bar.
The strongest competitors are BayesPrism and DWLS, which the breast-cancer benchmark ranked best on combined false positives and false negatives and on granular immune lineages. BayesPrism is the most dangerous because it already re-estimates a sample-specific malignant profile, a soft version of what trimming does. We will run both from their released implementations on identical pseudo-bulks, references and gene sets, with default settings plus the same hyperparameter grid size we allow ourselves.
There is no reported result on Lacuna for BayesPrism or DWLS RMSE or correlation on this benchmark; the file states only their ranking. We will therefore regenerate the benchmark's simulation protocol and report all numbers ourselves. Beating BayesPrism means lower normal-epithelial RMSE in the purity bins above 70%, with no loss of immune-lineage correlation at lower purity.
On the physical three-cell-line mixture, the reported figure to match is SCDC's Pearson $r = 0.991$ against true proportions (SCDC). With so little headroom, that dataset is a sanity check, not a discriminating test.
Four ablations isolate the ingredients: capped loss alone, capped-fraction penalty alone, a convex Huber loss in place of the capped loss, and non-negative least squares followed by thresholding and refitting.
Why not just…?
Why not just add a Lasso penalty to non-negative least squares? On the simplex the $\ell_1$ norm of the fractions is exactly one for every feasible solution, so a Lasso term changes nothing. This is arithmetic, not an empirical claim. Any penalty that zeroes small fractions under the sum-to-one constraint is non-convex, which is why the difference-of-convex structure is needed rather than ornamental.
Why not just threshold the non-negative least squares fractions and refit? This is a real threat to the sparsity half of the proposal. Post-hoc thresholding with refitting may recover most of the false-positive reduction. It is included as an ablation with its threshold tuned on the same grid as $\epsilon$. If it matches the capped-fraction penalty in false-positive/false-negative curves, we will report that the penalty adds a convergence guarantee but no accuracy. The burden then shifts entirely to the capped loss.
Why not just use a robust convex loss such as Huber, or DWLS-style gene weights? Huber loss still gives every badly mismatched gene a linearly growing influence. At high purity, the patient-specific malignant genes are numerous rather than a few outliers, and bounded influence is the property we expect to matter. The Huber ablation tests this directly. If Huber closes the purity-trap gap as well as capping, the non-convex loss is unnecessary and we will say so.
Why not just use BayesPrism, which already adapts the malignant profile per sample? It may already solve much of the purity trap. The benchmark's ranking suggests it is the best existing tool, yet the purity trap was reported across methods. The per-purity-bin comparison will show whether trimming adds anything beyond BayesPrism's adaptation. It will also show whether the two can be combined, by applying trimming to BayesPrism's updated profiles.
Proposed Experiments
The primary experiment regenerates the breast-cancer pseudo-bulk benchmark from the Wu et al. atlas: purities from 5% to 95%, ER+, HER2+ and TNBC subtypes, broad and granular lineages. The key change is a patient-held-out design, in which references are built from some patients and mixtures from others. Without that split, reference mismatch, the mechanism under test, would be absent. We also deconvolve mixtures built from the Bassez et al. and Pal et al. datasets using the Wu reference, as the benchmark did. Metrics are per-lineage RMSE and correlation by purity bin, and false-positive versus false-negative curves for rare lineages, traced by sweeping each method's sparsity knob.
The decisive check has two parts. Normal-epithelial RMSE above 70% purity must drop relative to BayesPrism. Trimmed genes must be enriched for true patient-versus-reference cancer differences. Accuracy without that enrichment would mean we are not fixing the mechanism we claim.
Secondary checks follow SCDC's real-data settings. On the three-cell-line mixture, we compare against $r = 0.991$. On the 77-donor pancreatic islet cohort, we test whether the beta-cell-fraction association with HbA1c survives and whether it is stable across references. SCDC reported $p = 0.0019$ with its ensemble, and single-reference methods were unstable. Finally, on the 1,038 TCGA breast samples, where there is no ground truth, we report only whether estimated normal-epithelial fractions in high-purity tumours shift in the direction the simulations predict.
References
- Performance of tumour microenvironment deconvolution methods in breast cancer using single-cell simulated bulk mixtures
- SCDC: Bulk Gene Expression Deconvolution by Multiple Single-Cell RNA Sequencing References
- DC Proximal Newton for Nonconvex Optimization Problems
- Distributed nonconvex optimization for sparse representation
- DC-LA: Difference-of-Convex Langevin Algorithm
- Enforcing Fair Predicted Scores on Intervals of Percentiles by Difference-of-Convex Constraints