MicroRNAs are short RNA regulators (~22 letters long) that silence genes by matching a 7-letter "seed" window at their front end — positions 2 through 8 — against target messenger RNAs. Because recognition is concentrated in that window, the system is brutally sensitive to where the enzyme Dicer cuts the precursor hairpin during maturation: a one-letter shift of the cut at the 5' end scrambles the seed entirely and redirects the microRNA onto a completely different set of targets, while a comparable shift at the 3' end barely matters. A single mismatch inside an established seed duplex can abolish repression, as viral suppression assays show directly (Cellular miR-130b inhibits replication of porcene reproductive and respiratory syndrome virus in vitro and in vivo). Meanwhile, the computational tools that predict these cut sites — from the Naive Bayes ranker MatureBayes (MatureBayes: A Probabilistic Algorithm for Identifying the Mature miRNA within Novel Precursors) to the duplex-level SVM of MiRduplexSVM (MiRduplexSVM: A High-Performing MiRNA-Duplex Prediction and Evaluation Methodology) — emit scores that are never checked for calibration. When MatureBayes was tested on newly cataloged mammalian precursors, exact boundary accuracy fell from roughly 28% to 15%, and nothing in the tool's output warned that it had left its competence zone.
The second thread of this proposal is a general machine-learning problem: uncertainty itself has no ground truth. There is no label saying how uncertain a prediction should be, so the field leans on proxies that are known to be biased. This proposal's central observation is that microRNA biogenesis accidentally solves the scarcity problem for this domain. Dicer cleaves real hairpins promiscuously, producing a cloud of boundary variants (isomiRs) that high-throughput sequencers sample by the millions — isomiR-SEA processed 144.2 million tags in 25 hours in its own benchmark (isomiR-SEA: an RNA-Seq analysis tool for miRNAs/isomiRs expression level profiling and miRNA-mRNA interaction sites evaluation). Each locus therefore comes with an empirical distribution of observed cut positions — effectively thousands of "annotations" that nobody had to pay for. This is exactly the ingredient that made supervised uncertainty learning possible in medical imaging, where four radiologists' disagreeing contours served as an aleatoric ground truth (Supervised Uncertainty Quantification for Segmentation with Multiple Annotations) — an ingredient unavailable almost anywhere else, but abundant here.
Core Claim
I claim that (a) small RNA-Seq read distributions over cleavage boundaries can serve as a large-scale, free aleatoric ground truth for boundary predictors, (b) calibration losses and ground-truth-free evaluation metrics must be re-derived for this task because its errors are structurally asymmetric — a 5' error scrambles the seed, a 3' error does not — and (c) a model trained with both ingredients will produce boundary predictions whose confidence provably tracks real failure, without anyone ever labeling an "uncertainty." No existing work does this: the biological tools output uncalibrated point predictions or margins, and the uncertainty-quantification literature — the AvUC training loss (Improving model calibration with accuracy versus uncertainty optimization), the multi-annotator supervised approach (Supervised Uncertainty Quantification for Segmentation with Multiple Annotations), and the ground-truth-free metrics UQ-AUC and UQ-C-index (Legitimate ground-truth-free metrics for deep uncertainty classification scoring) — has only ever assumed binary correctness on images, 2D Gaussians, or language QA. The distance structure of the error, and its functional asymmetry, is new territory.
Research Program
The first component is a boundary distribution model. Instead of predicting a single cut position, the model outputs a probability distribution $p_\theta(s \mid x)$ over candidate cleavage positions $s$ along a precursor hairpin $x$, for both the 5' and 3' ends. Features include position-wise nucleotide identity and base-pairing state (matched stem versus loop), letting us test a live contradiction in the literature: MatureBayes reports that structure states are needed to capture Dicer's "UUU" recognition signal, while MiRduplexSVM reports structure is redundant given a sequence-only encoding (MatureBayes, MiRduplexSVM). Both evaluated point accuracy only. My hypothesis is that both are partly right: sequence may suffice for the point prediction while structure governs the spread — and that is precisely a calibration question no one has run.
The second component turns read clouds into supervision. For each locus with small RNA-Seq coverage, the empirical histogram of observed 5' boundary positions $r(s \mid x)$ (extracted with isomiR-SEA's seed-first alignment, which already classifies iso_5p variants rather than discarding them) becomes the aleatoric target:
$$\mathcal{L}_{\text{alea}} = D_{\mathrm{KL}}\big(r(\cdot \mid x)\,\big\|\,p_\theta(\cdot \mid x)\big)$$
This is the direct analog of the supervised aleatoric loss in Supervised Uncertainty Quantification for Segmentation with Multiple Annotations, with sequenced reads replacing radiologists — but it needed no annotation budget and scales to every covered locus. Epistemic uncertainty is separated by variational dropout in the final layers, with the total predictive variance decomposing into the two parts by the law of total variance, exactly as in that paper. The result is a predictor that can say, per locus, "the data itself is ambiguous here" (wide $r$) versus "I have never seen a hairpin like this" (wide dropout variance over a sharp $r$) — a distinction the current tools cannot make and that matters, because a biologically promiscuous cutter and a confused model call for completely different downstream handling.
The third component fixes the loss and the metric. AvUC's four quadrants (accurate/certain, inaccurate/uncertain, etc.) are defined over binary correctness; here, correctness is graded and asymmetric. I define a seed-scramble severity weight $w(\hat{s}, s)$ that is maximal whenever the predicted and true 5' boundaries differ by even one nucleotide — because that changes the seed window and thus the target repertoire — and decays smoothly with 3' boundary distance, mirroring the interaction-zone asymmetry documented by isomiR-SEA's zone analysis (isomiR-SEA). This weight replaces the binary accuracy inside the AvUC quadrant penalties, and it also generalizes the ground-truth-free evaluation metric: seed-weighted UQ-AUC is the probability that a prediction with a correct seed receives lower uncertainty than one with a scrambled seed. The legitimacy argument carries over from the original metric's proof that UQ-AUC is maximized exactly when the score ranks the true misclassification probability (Legitimate ground-truth-free metrics for deep uncertainty classification scoring), but the binary indicator is swapped for the severity weight, and the proof must be re-established — that re-derivation is itself a deliverable of the proposal.
Finally, the evaluation protocol is designed around a known trap. Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results proves that when an uncertainty score and a correctness proxy share a confounder, AUROC rankings invert. The obvious proxy here — lexical seed-match overlap against a reference, as used in alignment pipelines — could share hairpin length or GC-content biases with any thermodynamic or compositional uncertainty signal. So all headline numbers use only held-out miRBase boundary labels under the ground-truth-free metrics, never lexical proxies, and a dedicated audit regresses uncertainty scores against length and GC content to detect confounding explicitly.
Proposed Experiments
Experiment 1 replicates and extends the MatureBayes evaluation protocol on miRBase (miRBase as used by MiRduplexSVM; the version-14 split design from MatureBayes): train on an earlier release, blind-test on precursors first annotated in a later release, exactly the regime where MatureBayes dropped from approximately 28% exact (65% within ±2 nucleotides) to 15% exact on 188 human and 141 mouse blind-test precursors. Primary metrics: exact and ±2-nucleotide accuracy (to show no point-accuracy regression), seed-weighted UQ-AUC (calibration), and a coverage–risk curve: if we abstain on the 20% of precursors the model flags as most uncertain, how much seed-scrambling error disappears? Success is a coverage–risk curve that dominates every baseline's, with the confidence interval on seed-weighted UQ-AUC excluding the best baseline.
Experiment 2 tests aleatoric calibration directly: the normalized cross-correlation between predicted per-locus boundary variance and held-out isomiR read variance, on RNA-Seq data disjoint from training. The benchmark to beat is 0.669, the NCC the supervised multi-annotator model achieved against radiologist disagreement on LIDC-IDRI (versus 0.597 for Kendall & Gal) (Supervised Uncertainty Quantification for Segmentation with Multiple Annotations). Matching that number using only free sequencing data would establish that read clouds are a genuine substitute for expensive multi-annotation.
Experiment 3 resolves the structure-versus-sequence contradiction as a calibration question: the same architecture trained with sequence-only, structure-only, and both feature sets, compared on point accuracy versus seed-weighted UQ-AUC and NCC. If structure is redundant for the point but necessary for the spread, the two papers' conflicting claims both become correct in their own regime — a clean, publishable resolution either way.
Experiment 4 measures whether boundary uncertainty propagates usefully downstream: predicted boundary distributions (rather than point boundaries) are fed into the target predictor TEC-miTarget (TEC-miTarget: enhancing microRNA target prediction based on deep learning of ribonucleic acid sequences) on its miRAW test set, where it reported up to +2.43% accuracy and +1.71% F1 over the best alternatives. The question is not whether target accuracy improves but whether target-set instability — how much the predicted target repertoire changes under boundary perturbation — shrinks when high-epistemic loci are flagged, given the demonstrated brittleness of Drosophila comparative-genomics target predictions to single-nucleotide 5' errors (Systematic discovery and characterization of fly microRNAs using 12 Drosophila genomes).
Baselines
The naive reference point is the canonical-boundary heuristic: always emit the boundary implied by the standard 2-nucleotide 3' overhang rule. It requires no model and defines the floor; MiRduplexSVM already showed rigid geometric rules misfit empirical cleavage and that learned predictors beat tools relying on them (MiRduplexSVM), so the heuristic mainly quantifies how much of the task is trivial. The strongest existing competitor is MiRduplexSVM itself — a polynomial-kernel SVM over a zero-padded binary sequence encoding predicting all four duplex boundaries, reporting up to a 60% accuracy increase over MatureBayes, MaturePred, MiRPara, and MiRdup on mammalian hairpins (no absolute accuracy value is reported on Lacuna for it, so the comparison will be re-run, not quoted). Its raw SVM margin will be used directly as an uncertainty score under identical splits, and it will be re-tuned with the same random-search budget and feature access as my model. A second calibrated-score baseline is the MatureBayes Naive Bayes posterior used as a confidence score — it is the one existing tool whose output is already a probability, and its published numbers (approximately 28% exact, 65% within ±2 nucleotides on training; 15% exact on the blind set) anchor the accuracy comparison. A third baseline is a deep ensemble of my own architecture trained on point labels: five independently initialized members, boundary-position variance across members as the uncertainty. This isolates whether the isomiR-supervised aleatoric loss and the seed-asymmetric severity weighting — the proposal's two key ingredients — add anything over generic ensemble disagreement at matched total compute (the ensemble gets the same GPU-hours split five ways). The final ablation is my full model with the isomiR loss removed and the severity weight collapsed to a binary correct/incorrect indicator, which separately tests each ingredient. Beating the baselines means: seed-weighted UQ-AUC strictly higher with non-overlapping bootstrap intervals on the blind split, NCC ≥ 0.669 in Experiment 2, and point accuracy within one percentage point of the best baseline, since a calibrated model that predicts boundaries worse is useless.
Why not just…?
Why not just use the scores the existing tools already output? MatureBayes already produces a posterior over candidate windows, so one could read it as confidence without building anything. The evidence against this is twofold. First, the paper itself treats the score as a heuristic ranking and reports no calibration; its blind-set collapse from 28% to 15% exact accuracy came with no detectable signal in the score, which is the failure mode this proposal exists to fix. Second, the broader literature documents that raw model probabilities are systematically overconfident near one even when wrong — the "softmax fallacy" (I Find Your Lack of Uncertainty in Computer Vision Disturbing) — and cross-entropy training drives them to extremes (Improving model calibration with accuracy versus uncertainty optimization). Still, this is a real threat to the proposal's necessity, which is why the existing scores appear as first-class baselines rather than being argued away: if the MatureBayes posterior or the SVM margin already achieves strong seed-weighted UQ-AUC on the blind split, the proposal's training contribution is dead and only the evaluation contribution survives.
Why not just collapse the isomiR reads to a consensus boundary and train normally? This is the cheapest use of the sequencing data and the alternative I take most seriously. The counter-evidence is direct: in the multi-annotator setting, training against a consensus mask was measurably worse than training against annotator variance — a Generalized Energy Distance of 0.346 versus 0.267 on LIDC-IDRI (Supervised Uncertainty Quantification for Segmentation with Multiple Annotations) — and collapsing reads throws away 5' variants that isomiR-SEA showed have entirely different target repertoires and non-random, age-varying distributions (isomiR-SEA). But the imaging result does not guarantee the same holds for one-dimensional cut positions, so the consensus variant is run as the headline ablation; if it matches on NCC and seed-weighted UQ-AUC, the aleatoric-supervision claim is wrong and the proposal shrinks to its evaluation component.
Why not just ensemble or temperature-scale a standard predictor? Five retrained copies give variance for free, and one scalar temperature fixes miscalibration post hoc on a validation set. Both are included as baselines rather than dismissed. The literature says temperature scaling degrades under distribution shift because it optimizes in-distribution likelihood only, and the ground-truth-free metrics paper shows that lowering Expected Calibration Error does not guarantee correct ranking of the true misclassification probability (Improving model calibration with accuracy versus uncertainty optimization, Legitimate ground-truth-free metrics for deep uncertainty classification scoring); the blind-release split is precisely a shift, so this is where the comparison will be decided. Ensembles are the honest threat: they capture epistemic doubt well. What they cannot do is decompose — ensemble variance conflates a genuinely promiscuous cutter with an unfamiliar hairpin, because nothing in their training ever saw the read distribution — so the decisive experiment is not overall UQ-AUC but the decomposition check: NCC against held-out read variance (aleatoric) and the divergence between dropout variance and ensemble variance on unfamiliar precursor families (epistemic). If ensembles match on both, the isomiR supervision adds nothing and I will say so.
Decisive Checks
Three findings would make the proposal. First, on the blind-release split, confidence scores from existing tools fail seed-weighted UQ-AUC while the proposed model passes, with the coverage–risk curve showing that abstaining on 20% of flagged loci removes most seed-scrambling errors — uncertainty you can act on. Second, NCC between predicted aleatoric variance and held-out read variance reaches roughly the 0.669 that expensive human multi-annotation bought in imaging, from data that already exists in public repositories. Third, the structure/sequence ablation settles the MatureBayes–MiRduplexSVM contradiction in calibration terms. Two findings would kill it, and the experiments are built to surface them early: existing scores already tracking failure on the blind split, or consensus training matching the variance-supervised model. Either negative result still leaves the seed-weighted, ground-truth-free evaluation protocol — the re-derived metric with its severity-asymmetric proof — as a standalone contribution the field currently lacks.
References
- Cellular miR-130b inhibits replication of porcine reproductive and respiratory syndrome virus in vitro and in vivo
- MatureBayes: A Probabilistic Algorithm for Identifying the Mature miRNA within Novel Precursors
- MiRduplexSVM: A High-Performing MiRNA-Duplex Prediction and Evaluation Methodology
- isomiR-SEA: an RNA-Seq analysis tool for miRNAs/isomiRs expression level profiling and miRNA-mRNA interaction sites evaluation
- Supervised Uncertainty Quantification for Segmentation with Multiple Annotations
- Improving model calibration with accuracy versus uncertainty optimization
- Legitimate ground-truth-free metrics for deep uncertainty classification scoring
- Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results
- TEC-miTarget: enhancing microRNA target prediction based on deep learning of ribonucleic acid sequences
- Systematic discovery and characterization of fly microRNAs using 12 Drosophila genomes
- I Find Your Lack of Uncertainty in Computer Vision Disturbing