Sequencers attach a quality score to every base they read, an estimate of how likely that base is wrong. These scores take up most of the space in raw sequencing files. Nanopore machines use 94 possible values per base, which makes the files hard to compress. Recent work showed these scores can be squeezed into 2 to 8 levels with almost no harm to genome assembly or human variant calling. Those tasks, however, average over 40 to 90 overlapping reads per position. Amplicon sequencing works differently. It reads one marker gene, such as 16S rRNA, from every microbe in a sample. Denoising tools then decide whether a rare read that differs from a common one by a single letter is a real strain or a sequencing mistake. That call often rests on 1 to 5 reads, and it depends directly on the error probabilities implied by the quality scores. Here coarse binning could plausibly erase the evidence, and no one has checked.
We propose to choose quality-score bins by what the denoiser decides, not by how faithfully the bins reproduce the original numbers. The bins are placed to keep the error probabilities that matter for rare single-letter variants, and they are weighted toward the bases where those decisions actually happen. Finding the bins is a one-dimensional weighted clustering problem, which can be solved exactly and cheaply. The resulting 2- or 3-bit quality alphabet serves two purposes. It saves storage, and it is small enough to let a fast context-counting denoiser use quality scores at all. Its authors explicitly left that extension undone.
Core Claim
Our claim is that the failure mode of quality quantization for amplicon data sits at low abundance and is invisible in average-case metrics. A quantizer built to preserve denoiser decisions can match full-precision rare-variant recall at 2–3 bits per score. Quantizers that only preserve the score distribution cannot. The closest work, Nanopore quality score resolution can be reduced with little effect on downstream analysis, tested fixed bins ($Q_2$, $Q_4$, $Q_8$), a constant score ($F_{10}$), and a context rule giving homopolymers finer bins. It evaluated these only on assembly polishing and diploid variant calling. Its conclusion that "coverage trumps precision" is exactly what should break when coverage per variant is one or two reads. Our work differs from it in three ways: the evaluation regime (amplicon denoising to exact sequence variants), the quantizer objective (driven by the downstream decision, not fixed bins), and the use of the quantized alphabet inside a denoiser.
Background and Roles of Prior Work
Amplicon sequence variant (ASV) methods replace the old practice of grouping reads at 97% similarity. Instead they model sequencing error and keep every exact sequence that is too abundant to be an error of a more common "parent" sequence. Several papers motivate why single-letter resolution matters. Thompson et al. used exact sequences as identifiers that can be compared across 27,751 Earth Microbiome Project samples. Lin et al. separated vaccine-derived strains from wild-type field strains. Bar Yosef et al. traced a sepsis cluster to a clinician through an exact Citrobacter freundii variant. These cases set the stakes: a quantizer that merges a rare real variant into its parent fails in precisely the setting that justified ASVs over 97% clusters.
DUDE-Seq supplies a second application. It treats the sequencer as a memoryless noisy channel. It counts the letters seen between flanking k-mer contexts and then decodes each base with Bayes' rule and a 4×4 substitution table. It is fast, but it ignores quality scores, and its authors name adding them as future work. With 94 raw quality values, conditioning the context counts on quality would split the counts so finely that each one becomes too noisy to use. A small quantized alphabet is what makes the extension feasible.
On the quantization side, we reuse ideas rather than models. Some methods cluster values with k-means: SKIM, Chen et al., and the activation-weighted k-means behind any4. Others fit Gaussian mixtures with expectation-maximization, as in Focused Quantization and BWA. These show that bins should follow the density of the data. Any4's practice of weighting clustering by downstream magnitude is the closest precedent for our importance weights. Companding warps values through a nonlinear map before uniform binning, as in Yamamoto's learnable companding and Tang et al.. It motivates our choice of quantization space: we bin in log error probability, a warp of the quality axis. Balanced quantization (Zhou et al.; Baskin et al.) makes bins equally likely to maximize entropy. It is our main foil. Equal-frequency bins put resolution where scores are common, but rare-variant decisions may hinge on scores that are uncommon. All of these methods minimize reconstruction or information loss on neural network weights. None targets a downstream statistical test.
Research Program
Let $q \in \{0,\dots,93\}$ be a raw quality value. Let $\hat\lambda(q)$ be the substitution error rate that the denoiser learns empirically for that value. We use the learned rate rather than the nominal Phred rate $10^{-q/10}$, since nanopore scores are often poorly calibrated. Let $w(q)$ be an importance weight: the number of times $q$ occurs at the differing bases of candidate sequences that are rare (below a threshold such as 10 reads) and one letter away from a more abundant sequence. These are the bases whose error rates decide the abundance test. A $K$-level quantizer partitions the quality axis into contiguous bins $b$. Each bin gets one representative error rate $\bar\lambda_b$. We choose bins to minimize
$$ \sum_{b}\sum_{q\in b} w(q)\,\big(\log\hat\lambda(q)-\log\bar\lambda_b\big)^2 , $$
which measures error in log space. The abundance test's p-value scales roughly with powers of $\lambda$, so its error is roughly proportional to the error in $\log\lambda$. Because the problem is one-dimensional with contiguous bins, the optimum comes from exact dynamic programming over at most 94 points, so no iterative clustering or initialization is needed. The same weights can be refit each sequencing run, since amplicon error profiles depend on the run.
We will build three things. The first is the quantizer: a weighted log-error variant, plus unweighted and Phred-space variants for ablation. The second is a harness that replaces the quality strings in FASTQ files and reruns standard statistical ASV denoisers (DADA2 and Deblur, the tools named in the file) unchanged. The third is a quality-conditioned DUDE-Seq, in which each context count and confusion table is indexed by the quantized quality of the center base. We will also run an analysis that predicts, from the size of the rare variant and the quantization error at its differing base, the abundance below which a variant is lost. We will then check that prediction against measured recall. To keep primer leftovers from creating spurious variants and confounding the comparison, we will standardize primer trimming with PIMENTO-inferred primers before any quantization.
Baselines
The naive reference points are full-precision quality scores, the ceiling, and the constant quantizer $F_{10}$ from Rivara-Espasandín et al., the floor. $F_{10}$ reduces the denoiser to an abundance threshold with a single error rate. The strongest existing competitors are that paper's $Q_4$ and $Q_8$ fixed quantizers and its homopolymer-aware $\langle Q_2,Q_8\rangle_\delta$. They are the only published quality quantizers shown to preserve downstream accuracy. On polishing the ZymoBIOMICS mock community with Medaka, $Q_4$ gave 174.17 mismatches per 100 kb against 175.56 for unquantized data. For ASV recovery under any quality quantizer, no reported result was found on Lacuna. That gap is the point of this proposal. We also include coarse vendor-style interval binning of Illumina scores, for which the file gives no citation, and balanced equal-frequency bins following Zhou et al.. Two ablations isolate our ingredients. Unweighted k-means in Phred space, in the style of SKIM and any4, removes both the log-error warp and the decision weights. Unweighted clustering in log-error space removes only the weights. All quantizers get the same number of levels $K\in\{2,4,8\}$. They are fit on the same held-out calibration fraction of each run and fed to the same denoiser version with identical settings and the same tuning budget. Storage is measured with the same back-end compressor (gzip, as in the prior work). We will count it as beating the baselines if the method keeps rare-variant recall at 1–5 reads within the run-to-run noise of full precision at $K=4$ while $Q_4$ and balanced bins do not, and if it does so without adding false ASVs.
Why not just…?
Why not just use $Q_4$, which already preserved polishing accuracy? It may suffice, and this is the main threat. If $Q_4$ matches full precision on rare-variant recall, the honest conclusion is that the "coverage trumps precision" result carries over to amplicons, and the elaborate quantizer is unnecessary. The abundance-stratified recall experiment below settles the question directly. The prior evidence for $Q_4$ comes from 40–90x consensus tasks, which are structurally unable to show a low-abundance failure.
Why not just discard quality scores and set them to a constant? $F_{10}$ saved nearly 70% of storage in Rivara-Espasandín et al.. But a constant score turns the denoiser's position- and run-specific error model into one global rate. We expect this to inflate false variants in regions with high error and suppress real ones in regions with low error. The $F_{10}$ arm measures exactly this. If it performs as well as full precision, quality scores carry no information for ASV calling at our test depths, and that would be a notable negative result.
Why not just use equal-frequency bins, the textbook distribution-aware quantizer? Equal-frequency bins maximize the information kept about the score itself, as Zhou et al. argue. They also acknowledge that equalizing distorts what is downstream. Common high scores would get most of the bins, yet a rare variant's fate may depend on how a low-score base at its differing position is treated. Comparing balanced bins against our weighted variant at equal $K$ tests whether the downstream weighting matters, separately from simply following the data distribution.
Why not just go back to 97% clusters if compute is the issue? The studies above, from Thompson to Bar Yosef, show that clustering destroys exactly the strain resolution at stake. We include clustering only as a reference for how much resolution a lossy pipeline gives up.
Proposed Experiments
The decisive experiment uses mock communities with known exact sequences: the ZymoBIOMICS standard sequenced as full-length 16S on nanopore, and the synthetic and 454 mock sets used by DUDE-Seq. To probe low abundance in a controlled way, we spike in real variants computationally. We take reads of one strain, introduce a single substitution at a chosen position, and keep each read's original quality string, so the variant inherits realistic quality patterns. We then subsample the variant to 1, 2, 5, 10, and 20 reads. We report recall of known variants against abundance for every quantizer and every $K$, together with false ASV counts and bits per quality score after compression. The main falsification test is whether $Q_4$ and balanced bins lose recall at 1–5 reads while the weighted quantizer does not. If no quantizer differs from full precision, the core claim is false.
The second experiment tests the DUDE-Seq extension on the same mocks. We compare original DUDE-Seq, quality-conditioned DUDE-Seq on raw scores, and quality-conditioned DUDE-Seq with each quantizer at $K=2,4$, measuring per-base error reduction. DUDE-Seq reports up to 23.8% reduction over raw reads. A conditioned version on quantized quality that beats this number on the same data would be the first quality-aware result for this denoiser. We expect raw-quality conditioning to underperform because each count is spread across too many values. That would show quantization doing statistical work, not just saving storage.
The third experiment checks scale and ecology on a subset of the Earth Microbiome Project data. We measure storage saved, and we check whether quantized and full-precision pipelines give the same ASV tables, the same diversity between samples, and the same accuracy for a random forest predicting environment type. Thompson et al. report 91% accuracy on the coarse environment split and 84% on the fine-scale one. Matching those numbers would show that rare-variant protection does not cost community-level conclusions. Finally, we compare the analytically predicted abundance threshold for losing a variant with the measured threshold. A good match would let practitioners choose $K$ for a target sensitivity without rerunning benchmarks.
References
- Nanopore quality score resolution can be reduced with little effect on downstream analysis
- A communal catalogue reveals Earth’s multiscale microbial diversity
- Broad-spectrum respiratory tract pathogen identification using resequencing DNA microarrays
- Outbreak of Sepsis Following Surgery: Utilizing 16S RNA Sequencing To Detect the Source of Infection
- DUDE-Seq: Fast, flexible, and robust denoising for targeted amplicon sequencing
- SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization
- Channel-Wise Mixed-Precision Quantization for Large Language Models
- any4: Learned 4-bit Numeric Representation for LLMs
- Focused Quantization for Sparse CNNs
- Achieving binary weight and activation for LLMs using Post-Training Quantization
- Learnable Companding Quantization for Accurate Low-Bit Neural Networks
- Efficient Representation of Large-Alphabet Probability Distributions
- Balanced Quantization: An Effective and Efficient Approach to Quantized Neural Networks
- UNIQ: Uniform Noise Injection for Non-Uniform Quantization of Neural Networks
- PIMENTO: A PrIMEr infereNce TOolkit to facilitate large-scale calling of amplicon sequence variants