Telling Real lncRNA Sponges from Bystanders with Exponent-Coded Information Decomposition

Many long non-coding RNAs (lncRNAs, RNA transcripts longer than 200 nucleotides that make no protein) are thought to act as "sponges." A sponge soaks up a microRNA, a short RNA that normally silences messenger RNAs, and so frees that microRNA's targets. Well-studied cases include SNHG12 freeing HIF1α from miR-199a-5p and NEAT1 freeing CDKN1B from miR-222-3p. Finding sponges across the genome, however, still relies on co-expression statistics. These statistics cannot separate two different situations. In the first, the lncRNA really changes how strongly the microRNA represses the target. In the second, the lncRNA and the target simply rise and fall together.

Partial information decomposition (PID) splits the information that several sources carry about a target into three kinds of parts: unique (only one source has it), redundant (each source has it on its own), and synergistic (it appears only when the sources are read jointly). That split is the right instrument for this problem, because a working sponge should create synergy among lncRNA, microRNA and target. Two things block its use. Exact PID is too expensive to run over millions of candidate triplets, and the scalable versions assume Gaussian data, while lncRNA counts are sparse, full of zeros, and heavy-tailed. We propose to borrow the central trick of lossless weight compression to make exact discrete PID cheap and appropriate for count data. That trick is to notice that floating-point exponents carry only 2–3 bits of real information, and then to code exactly that information.

Core Claim

The weight-compression literature shows that floating-point exponents are badly under-used. DFloat11 finds that only about 40 of the 256 possible BFloat16 exponents appear often, which gives an exponent entropy of roughly 2.6 bits. ECF8 reports 2–3 bits of exponent entropy in FP8 weights and attributes it to heavy-tailed, α-stable value distributions. Both papers treat this as a way to save memory.

We use the same structure as a discretization principle for transcript abundances, which are also heavy-tailed. Each expression value is coded by its base-2 exponent. Zero gets a dedicated symbol, in the way a float format reserves special codes. Rarely used exponents are merged, Huffman-style, until the alphabet has at most $K$ symbols. Concretely, for a count $x$ the code is

$$c(x) = \begin{cases} 0 & x = 0 \\ \min\!\big(K-1,\ \max(1, \lfloor \log_2 x \rfloor - b + 1)\big) & x > 0 \end{cases}$$

Here $b$ is a per-gene offset chosen so that the populated exponents fill symbols $1$ to $K-1$.

With $K \le 8$, the joint distribution of a triplet (lncRNA $L$, microRNA $M$, target mRNA $T$) has at most 512 cells. On a support that small, an exact two-source PID solved by convex optimization costs milliseconds. Two standard identities then give a cheap screen and a sanity check. Here $UI_L$ is the information about $T$ unique to $L$, $RI$ is redundancy, and $SI$ is synergy:

$$I(L;T \mid M) = UI_L + SI, \qquad I(L;M;T) = RI - SI.$$

The first identity shows that the conditional mutual information computed by JAMI mixes unique and synergistic information together, while also bounding synergy from above. Triplets with low conditional mutual information can therefore be dropped before any optimization runs.

The specific new claim has two parts. First, synergy computed on exponent codes ranks experimentally supported sponge triplets above co-regulated bystanders better than conditional mutual information, interaction information, or Gaussian PID do. Second, it does so at genome scale.

Research Program

The work has three parts.

Measuring the exponent statistics. We first test whether the analogy to weights holds for transcript data. We will quantify transcripts with Salmon and Kallisto, because the lncRNA quantification benchmark shows that these pseudoalignment tools detect about 16% of annotated lncRNAs versus 10–13% for HTSeq. HTSeq-style counts would therefore build undercounting into the codes. On these quantifications we will measure, per gene, the empirical entropy of the exponent field. We will also measure how much of the pairwise mutual information between genes survives once the mantissa (the within-exponent detail) is dropped. If exponents hold most of the shared information, the code throws away mainly noise. If they do not, the premise fails early and cheaply.

Building the pipeline. The pipeline has four steps. It codes all genes, forms candidate triplets, screens them by conditional mutual information on the codes, and solves an exact two-source PID on the triplets that survive. Candidate triplets come from experimentally validated lncRNA–microRNA pairs in lncRNASNP2 (15,386 base pairs, used by TEC-LncMir), crossed with mRNAs that co-vary with the lncRNA. Because the code alphabet is fixed and small, many triplets produce identical or near-identical joint histograms. We will cache solutions by histogram to cut cost further.

Positioning against prior PID methods. Different prior methods play different roles here.

Gaussian PID brings the cost down to quadratic in dimension under a Gaussian assumption. Its bias correction stays stable with as few as 100 samples. It motivates our sample-size regime but constrains our design, since zero-inflated counts break its assumption.

Analytical PID for affine systems gives closed forms for Poisson and other count families. It assumes a chain-like affine dependence. MicroRNA titration by a sponge is not affine, so we use it only as a check on simulated affine cases.

SΩI estimates only the net balance of redundancy minus synergy. For three variables this collapses to interaction information, which we test directly as a baseline.

The redundancy-lattice analysis shows the number of atoms exploding with more sources. That result motivates our restriction to the two-source triplet, where there are only four atoms.

MELON motivates the bet that synergy matters in omics. Its baselines that ignored the decomposition fell to chance on a synergy simulation. MELON uses PID as a training objective for tissue-domain learning and never computes PID atoms for individual regulatory triplets, which is what we do.

IIMLP uses sequence entropy features only to separate lncRNAs from mRNAs. It does not address what a lncRNA does.

Baselines

Naive reference. The naive reference is Pearson and partial correlation between lncRNA and target given the microRNA. This is the default ceRNA ("competing endogenous RNA," another name for sponge activity) screen. It will be computed on the same Salmon log-abundances and the same candidate triplets. Our method must beat it on ranking metrics against validated triplets, or there is nothing to report.

Strongest existing method. The strongest existing method is JAMI's conditional mutual information $I(L;T\mid M)$, computed with its adaptive partitioning and zero handling. It is the established tool built specifically for this ceRNA question, it is fast, and it already handles zeros. JAMI reports an approximately 70-fold speed-up over CUPID on TCGA breast cancer data. We found no reported result on Lacuna for JAMI's accuracy at recovering validated sponge triplets, so we will run JAMI ourselves on identical data, triplets and sample splits. Our method beats it if PID synergy gives higher area under the precision–recall curve on held-out validated triplets. It must also separate simulated sponge from co-regulation regimes in cases where conditional mutual information values coincide.

Further comparisons. We also compare against interaction information on the same codes, and against Gaussian PID on log1p-transformed abundances using its published bias correction. Both get the same samples. Gaussian PID gets the same wall-clock budget as our solver.

Ablation. The ablation that isolates our key ingredient keeps everything identical but swaps the exponent code for equal-frequency (quantile) bins with the same $K$, and for the code with the zero symbol removed. If quantile bins do as well, the borrowing from weight compression adds nothing, and we will say so.

Why not just…?

Why not just compress the weights of a lncRNA transformer so it can read full-length transcripts? This is the obvious way to combine these directions. TEC-LncMir truncates sequences at 25 kb because of GPU memory limits. But its memory goes to length-dependent activations and a contact map that grows with the product of the two sequence lengths, not to the parameters. Lossless exponent coding saves about 30% of weight memory, per DFloat11. That is the wrong part of the memory budget, and activation checkpointing or chunked attention would do more. We therefore dropped this idea.

Why not just use JAMI's conditional mutual information? By the identity above, it equals unique information plus synergy. A lncRNA that tracks the target for reasons unrelated to the microRNA scores just as high as a true sponge. This alternative is a real threat on real data, because validated triplets might also carry high unique information. The held-out ranking comparison and the simulation, where the ground-truth regime is known, settle the question directly.

Why not just compute interaction information, which is free? It equals redundancy minus synergy, so a triplet that has both can score near zero. Our simulation includes mixed regimes designed to expose exactly this cancellation. If real triplets never fall in such regimes, interaction information may be enough, and the comparison will show it.

Why not just apply Gaussian PID to log counts? It scales well and is the most serious competitor on speed. It is, however, a conjecture-based approximation with a heuristic bias correction, and log1p does not remove the point mass at zero. We expect it to fail most on low-abundance lncRNAs. We will report performance stratified by zero fraction to test exactly that.

Proposed Experiments

Simulation. The first experiment is a mass-action titration simulation. A microRNA is shared between a lncRNA and an mRNA, with a known sponge strength, and we add negative-binomial noise with zero inflation. Matched co-regulation controls give both RNAs a common upstream driver and no microRNA competition. Sample sizes span 100 to 1,000. We report the receiver operating characteristic and area under the precision–recall curve for separating the two regimes, for every method.

Real data. The second experiment uses TCGA breast cancer microRNA and mRNA data, as in JAMI, plus the 210-sample pan-cancer set requantified with Salmon. Positives are lncRNASNP2-supported lncRNA–microRNA pairs completed with their reported targets, including the curated axes above: SNHG12/miR-199a-5p/HIF1α, HOTAIR/miR-519b-3p/PRRG4 and NEAT1/miR-222-3p/CDKN1B. Negatives are matched co-expressed triplets that lack microRNA support. We measure ranking quality, cross-cancer stability of the synergy scores, and wall-clock time per million triplets against JAMI and Gaussian PID on the same hardware.

Decisive checks. Three outcomes decide the project. First, if exponent codes do not preserve most of the pairwise mutual information, the representation premise fails. Second, if quantile bins match exponent codes, the contribution reduces to "use discrete PID." Third, if conditional mutual information ranks validated triplets as well as synergy does, PID is unnecessary for this question. Each outcome is informative, and only the conjunction of all three favourable results supports the full claim.