Precision Follows the Read: Low-Bit Host Filtering That Escalates Doubtful Reads

In a clinical metagenomic sample, more than 99% of the DNA comes from the patient, not the pathogen (Profiling of tumour-associated microbiota in human hepatocellular carcinoma; High Sensitivity of Shotgun Metagenomic Sequencing in Colon Tissue Biopsy by Host DNA Depletion). Yet a single microbial read can settle a diagnosis (Adjunctive Diagnosis of Mycobacterium gordonae Pneumonia by Metagenomic Next-Generation Sequencing). The first computational step in such a pipeline is to throw away human reads. That step therefore has two jobs at once. It must be very cheap, because it touches every read. It must also almost never discard a pathogen read, because those reads are rare and precious. A small neural network can do this filtering without a multi-gigabyte reference genome (AMAISE). The obvious way to make such a network cheaper is to quantize it, meaning to store and compute its numbers in fewer bits. Shrinking a model, however, does not by itself make it run faster on real chips. And a quantized filter that holds up on average could still quietly drop the one read that mattered.

We propose to apply mixed precision across reads instead of across the network's internal channels. Every read first passes through a uniformly low-bit filter, at INT4 or below. Only reads the filter is unsure about are sent to the same network running at full or 8-bit precision. Because host reads dominate the sample and are mostly easy, the expensive tier sees a small fraction of the data. Because each tier runs one uniform precision, the hardware runs regular, well-supported kernels. This sidesteps the packing and irregular memory access overheads that keep channel-wise mixed precision from producing wall-clock speedups.

Core Claim

Mixed-precision research has found that assigning different bit-widths within a tensor breaks uniform memory layouts and adds bit-manipulation work, which erases the expected gains. The file traces this through Dustin, which reports software packing and unpacking costing 1.9× to 7.7× relative to native support, and through channel-wise schemes such as Channel-Wise Mixed-Precision Quantization for Large Language Models and SKIM. Cherry on Top keeps about 0.39% of weights in FP16 and notes that turning this into speed needs specialized kernels.

Metagenomic host depletion offers a different place to put the heterogeneity: the sample, not the tensor. Our claim has three parts. First, a quantization-aware-trained low-bit host filter remains well calibrated on the overwhelming majority of reads. Second, its errors concentrate on a small set of identifiable reads, such as low-complexity sequence, unusual GC content, and divergent or novel taxa. Third, a confidence-gated escalation to a high-precision tier recovers full-precision sensitivity on rare pathogen reads while keeping most of the low-bit throughput.

If $c_{lo}$ and $c_{hi}$ are the per-read costs of the two tiers, and $\rho$ is the fraction of reads escalated, the cascade's per-read cost is

$$c_{cascade} = c_{lo} + \rho\, c_{hi}.$$

The speedup over the high-precision model alone therefore approaches $c_{hi}/c_{lo}$ when $\rho$ is a few percent. The idea fails if quantization makes the model confidently wrong on pathogen reads, because escalation cannot catch errors the filter does not doubt. The first thing we test is whether that happens.

Research Program

The workload is an AMAISE-style host filter: a 4-layer one-dimensional convolutional network (a network that slides learned filters along the sequence) with global average pooling over variable-length reads. AMAISE reports above 98% accuracy and a 0.003 GB footprint (AMAISE). We retrain this architecture with quantization-aware training at INT8, INT4, ternary and binary weights. Quantization-aware training means simulating low precision during training so the model adapts to it; we use the straight-through estimator approach that Cherry on Top uses for language models. The training data are host genomes (human, pig, mouse), microbial genomes, and real nanopore reads, mirroring AMAISE's training mix.

The model is small, so we also test whether its weights show the "few parameters matter most" pattern that Cherry on Top found in transformers. That paper explicitly leaves open whether the pattern holds for sequence CNNs.

Two co-design details follow from the data rather than from language-model practice. The input is a four-letter one-hot alphabet, so the first convolution over a window of $k$ bases equals a lookup into a $4^k$-entry table. That layer becomes an exact table read at any precision, in the spirit of the lookup-table execution in FluxBin, but without its approximation. Variable read lengths also cause irregular memory access, so we bucket reads by length before batching so that kernels see contiguous, padded-free tiles.

The cheap tier runs uniform low-bit kernels. These are standard INT8 and INT4 paths on a commodity GPU, plus bit-packed sub-byte paths on a laptop-class CPU of the kind that sits beside a portable sequencer. The escalation tier runs the FP16 or INT8 model.

The escalation rule is a threshold on the cheap tier's host probability. It is tuned so that a fixed host-removal rate is met while spike-in pathogen reads are retained. We additionally test a calibrated variant that uses temperature scaling on held-out data. Hardware like Dustin motivates sub-byte execution, but we cannot fabricate it. All speed and energy claims will therefore be measured on hardware we actually run. Precisions that give no wall-clock gain on that hardware will be reported as such.

Baselines

The naive reference point is no host filter at all. All reads go straight to a reference-based classifier: Kraken2 for accuracy, and its GPU counterparts cuCLARK (Accelerating metagenomic read classification on CUDA-enabled GPUs) and MetaCache-GPU (MetaCache-GPU) for speed. For context, cuCLARK reports up to 50 million reads per minute on a Titan X. MetaCache-GPU reports queries 1.9 to 6.2 times faster than Kraken2 on RefSeq 202, at the cost of a 74 GB index. These tools set the memory and throughput bar any neural filter must justify itself against. We run them on the same reads and hardware.

The strongest direct competitor is full-precision AMAISE, retrained by us on the identical data split with the same tuning budget. Its reported figures are above 98% host/microbe accuracy, a 0.003 GB model, a 14–18% reduction in downstream memory, and under 20 minutes on high-host samples (AMAISE). The cascade beats it if two things hold: it matches its recall on spike-in pathogen reads, within the confidence interval at equal host removal, and it gives at least 2× higher reads per second or reads per joule on the same device.

A second strong competitor is uniform INT8 AMAISE, the off-the-shelf choice that SmoothQuant shows to be hardware-friendly on Tensor Cores (up to 1.56× throughput over FP16 for language models). No reported result was found on Lacuna for quantized host-depletion networks at any bit-width. Our INT8 numbers will be the first, and the cascade must beat INT8 on throughput at equal pathogen recall to be worth its complexity.

The key ablations separate the ingredients. The low-bit filter alone, without escalation, tests whether escalation is needed at all. A channel-wise mixed-precision filter with the same average bit-width, in the style of SKIM, tests whether placing heterogeneity per read rather than per channel is what yields real speed. A cascade whose cheap tier is a smaller full-precision CNN rather than a quantized one tests whether low precision, as opposed to cascading in general, is the source of the gain.

Why not just…?

Why not just use uniform INT8 and stop? This is the most serious threat. The network is tiny, and INT8 may already lose nothing while running fast enough for a single benchtop sequencer. If our measurements show that INT8 matches FP16 pathogen recall, and that sub-byte kernels give under about 1.5× real speedup on available hardware, we will report that the cascade is unnecessary at that scale. Its case then rests on throughput-bound settings such as the 1.5 trillion base-pair wastewater stream behind METAGENE-1, or on battery-bound portable devices. The INT8 baseline and the reads-per-joule comparison settle this directly.

Why not just use reference-based host subtraction or k-mer classifiers? Alignment-based subtraction as in SURPI (A cloud-compatible bioinformatics pipeline) and GPU k-mer tools are fast but carry tens to over a hundred gigabytes of index. The GPU tools report memory capacity as their bottleneck, and MetaCache-GPU needs 151 GB for AFS31+RefSeq202. Exact k-mer matching also degrades on noisy long reads. On a workstation with ample RAM these tools may well win on raw speed, and we will say so. The comparison that matters is reads per second per gigabyte of resident memory, together with pathogen recall on noisy nanopore reads.

Why not just use within-network mixed precision like CherryQ or SKIM? These methods keep sensitive weights or channels at high precision. They may give the best accuracy per bit, but their own authors and the hardware literature indicate that the irregular layouts block wall-clock gains without custom kernels (Cherry on Top; Dustin). The equal-average-bit ablation measures both accuracy and wall-clock speed, so it shows whether per-read heterogeneity really converts into speed where per-channel heterogeneity does not.

Why not just deplete host DNA in the wet lab? Detergent lysis with nuclease digestion and microfluidic filtration (Microfluidics-Based Enrichment) reduce host load but do not remove it, and they add preparation time. The filter is complementary to them, not a replacement. We will include a wet-lab-depleted sample to report how the escalation fraction $\rho$ shifts when host content falls.

Proposed Experiments

The first decisive check is calibration under quantization, run before any speed work. We spike reads from held-out pathogens, including viruses, fungi and divergent bacteria, into human backgrounds at abundances down to a single read per million. We then measure where the low-bit filter's misclassified pathogen reads fall in its confidence distribution. If more than a small fraction of lost pathogen reads are confidently called host, escalation cannot rescue them and the claim fails at that bit-width.

The second check is the operating curve. For each bit-width, we plot pathogen-read retention against host-removal rate and escalation fraction $\rho$ on AMAISE's human, pig and mouse benchmarks. We also run on CAMI II data (Critical Assessment of Metagenome Interpretation: the second round of challenges), using its clinical pathogen sample and its plant-associated sample, whose eukaryotic content stresses a host/non-host boundary. A useful operating point retains essentially all pathogen reads while escalating only a few percent of reads.

The third check is wall-clock reality. We measure reads per second, reads per joule and peak memory for each tier and for the cascade on a commodity GPU and a laptop-class CPU, with and without length bucketing and the first-layer lookup table. These runs show which co-design choices carry the speedup.

The final check is end to end. We feed the filtered reads into Kraken2 and record time-to-correct-identification on the CAMI II clinical sample and on spike-ins. The goal is to show that the faster filter shortens the time to a diagnosis without removing the reads that make that diagnosis possible.