A surgeon on a tablet, or a radiologist on a congested hospital link, wants to rotate and cut through a 3D CT scan immediately. They do not want to wait minutes for a multi-hundred-megabyte, 16-bit volume to arrive. Today the choice is between two bad options. One is to wait for the full lossless file. The other is to look at a blurred or neurally approximated stand-in, and nothing tells the viewer which parts of the picture are wrong. We propose a third option. The server sends a coarse volume first, then sends the exact prediction errors (residuals) in most-significant-bit-first order. At every moment, each voxel therefore has a known interval that must contain its true value. The client's GPU ray caster pushes these intervals through the rendering math, so for every screen pixel it knows a guaranteed range of possible colors. Once that range falls inside one display gray level, the pixel is provably identical to what the full lossless scan would show for this view and this transfer function (the mapping from tissue density to color and opacity). The client then asks the server for whichever residual bits will shrink the remaining uncertainty fastest for what is on screen right now.
In two sentences: progressive residual streaming plus interval arithmetic in the ray caster lets the viewer certify, pixel by pixel, when the rendering is already exact. It also lets the viewer request only the residual bits that matter for the current view. The scientific bet is that "exact on screen" arrives far sooner than "exact on disk," and that this gap is large, measurable, and safe.
Core Claim
Direct volume rendering accumulates color along each viewing ray by front-to-back blending, $C=\sum_i c_i\alpha_i\prod_{j<i}(1-\alpha_j)$. Here $c_i\in[0,1]$ and $\alpha_i$ are the color and opacity that the transfer function assigns to the $i$-th ray sample. Suppose each sample's density is only known to lie in an interval. Hardware trilinear interpolation gives a convex combination of eight voxels, so the interval propagates exactly through it. One can then precompute, for any density interval, the range of $\alpha$ and of $c\alpha$ that the transfer function can produce. This works like the min-max tables used in empty-space skipping. Perturbing one sample's opacity changes its own term by at most its weighted range, and rescales the remaining tail, whose total is at most the transmittance reaching that sample. Summing over samples gives a conservative bound on the width of the pixel's possible color range:
$$W \;\le\; \sum_i \Big(\prod_{j<i}(1-\alpha^{\mathrm{lo}}_j)\Big)\big(\Delta^{c\alpha}_i+\Delta^{\alpha}_i\big),$$
In this bound, $\alpha^{\mathrm{lo}}_j$ is the smallest opacity sample $j$ can take, and $\Delta^{c\alpha}_i$, $\Delta^{\alpha}_i$ are the ranges of $c\alpha$ and $\alpha$ over sample $i$'s density interval. A pixel is certified when its lower and upper color bounds round to the same 8-bit value.
The bound itself is elementary. What is new is building a streaming codec and a scheduler around it. Three facts make the bound useful in practice. First, residuals after a decent prediction are tightly concentrated near zero, so after a few bit-planes most voxels have narrow intervals. Second, transfer functions are flat over most of the density range (air, fat, or soft tissue windowed to a constant), so huge regions produce zero uncertainty even with coarse intervals. Third, opaque tissue in front drives the transmittance factor toward zero, so voxels behind it barely matter. The claim to test is this: on clinical CT and MRI, the bytes needed to certify 99% of pixels for a typical view are a small fraction of the full lossless file, and a view-driven request order reaches that point substantially faster than any view-agnostic progressive codec.
Research Program
The work has three pieces: the codec, the renderer, and the measurement harness.
The codec borrows its residual representation from Practical Lossless Volumetric Medical Image Compression via Tri-plane Context Tree Learning. That paper splits 16-bit prediction errors into a zero flag, a sign flag, and most- and least-significant parts to avoid huge symbol alphabets. It is a strong offline CPU lossless codec with no rendering or streaming path. We change it in two ways. We cut residuals into fixed-size 3D bricks, so the client can request them independently. We send each brick's magnitude bit-planes most-significant first, so that after $k$ planes every voxel's true value lies in an interval of width $2^{b-k}$, where $b$ is the residual bit depth.
The prediction must be non-causal. It is computed from the transmitted base level (a 4× downsampled volume) plus inter-slice linear prediction from that base, never from neighbors' not-yet-exact values. This keeps each voxel's interval independent of what has arrived elsewhere. It will cost some compression relative to TCT's causal context trees, and measuring that cost honestly is part of the program. The predict-then-code-the-sparse-error pattern follows A Novel Progressive Image Scanning and Reconstruction Scheme based on Compressed Sensing and Linear Prediction and Learned Lossless JPEG Transcoding via Joint Lossy and Residual Compression. The latter also warns that residuals can dominate a hybrid bitstream. That warning is exactly why the residuals should be sent in an order that matters for the viewer.
The renderer is a CUDA ray caster that stores, per brick, the current midpoint estimate and the interval half-width. It renders the midpoint image for display and, in the same pass, accumulates the bound above per pixel. Brick decoding happens on the GPU from bit-plane data, in the spirit of the dense/sparse block routing in MMVC. Bricks whose remaining planes are all zero skip work entirely.
A scheduler on the client estimates, for each brick, how much summed pixel uncertainty its next bit-plane would remove per byte. It uses the per-brick contribution already accumulated during the ray pass, and sends a priority list upstream. When the user changes view or transfer function, the certificate is recomputed from what has already arrived. Nothing is ever thrown away, and the stream always converges to the bit-exact lossless volume.
Shading complicates the guarantee, because gradients are computed by central differences from neighbors. Intervals do propagate through differences, but they widen. We will certify unshaded emission-absorption rendering first, then report how much later shaded pixels certify. We will not overclaim the shaded case.
The harness addresses the fourth direction directly. Guarantees in this area, as in A Local Clustering Algorithm for Massive Graphs or A Subquadratic Algorithm for Computing the L₁-Distance Between Two Terrains, are often stated and never profiled. We will measure wall-clock time on real GPUs under emulated bandwidth-limited links, not just bits per voxel. We will also check the certificate empirically: render the lossless volume and confirm that no certified pixel ever differs.
Baselines
The naive reference is to transfer the whole volume losslessly and then render. We use the best lossless codec available, TCT Practical Lossless Volumetric Medical Image Compression via Tri-plane Context Tree Learning, run on the same volumes. Time to first exact frame equals transfer time plus decode time. TCT reports decoding 40 to 145 times faster than DNN-based competitors and model overhead under 0.15% of file size. We found no reported compression ratio on Lacuna for TCT on CHAOS, MosMedData, or Trabit, so we will measure its bits per voxel ourselves. Beating this baseline means reaching 99% certified pixels in clearly less wall-clock time at every bandwidth tested. It also means paying no more than a modest premium in final lossless size, which we will report rather than hide.
The strongest streaming competitor is MIST's progressive High-Throughput JPEG 2000 path Towards Resource-Efficient Streaming of Large-Scale Medical Image Datasets for Deep Learning. It reports an average 56.22% storage and transmission reduction across eight datasets, 96.10% transmission reduction at the smallest sub-resolution with SSIM 0.83 ± 0.06, and exact reconstruction at full resolution. It is the strongest because it is already lossless-at-the-end, progressive, and deployable. We will stream its slice codestreams resolution-first into a GPU 3D texture with the same ray caster, at the same bandwidths, on volumes drawn from datasets it already covers (MSD-Liver, AMOS, LIDC-IDRI). We will also derive the tightest per-voxel error interval its truncated wavelet layers admit, so it gets a certificate too. Beating it means fewer bytes to 99% certified pixels for matched views and transfer functions.
The strongest lossy rendering competitor is fV-SRN Fast Neural Representations for Direct Volume Rendering. It reports rendering up to 165× faster than neurcomp and compression ratios up to 1:10,000. Alongside it we use Instant Neural Representation for Interactive Volume Rendering, which reports 10–60 FPS at PSNR above 30 dB. We will train each per volume under a fixed GPU-time budget, include that training time in time-to-first-frame, and measure the fraction of pixels matching the lossless rendering, which these methods cannot certify.
Two ablations isolate the key ingredients. One keeps the codec and renderer but sends bit-planes in a fixed view-agnostic order (all bricks, plane by plane). This tests whether the view-driven scheduler is what matters. The other swaps our downsampled base for an fV-SRN base with residuals coded against it. This tests whether a better but costlier predictor pays for its training latency.
Why not just…?
Why not just stream HTJ2K progressively, as MIST does? This is the real threat. MIST already delivers exact reconstruction at the end and very cheap previews. Its order, however, is fixed per slice and independent of what the viewer is looking at. Its truncation error is controlled in an average sense, not as a per-voxel bound. Without per-voxel bounds the renderer cannot tell the user which pixels are already final. The head-to-head comparison of bytes-to-certification, with MIST given its best derivable bound, settles it. If MIST matches us within noise, the contribution shrinks to the certificate layer alone, which could sit on top of HTJ2K, and we would say so.
Why not just send a downsampled volume and let GPU trilinear filtering fill in? That is literally our first layer, so the question is whether anything after it is needed. MIST's own numbers show SSIM falling to 0.83 at low resolutions. The CHAOS challenge CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentation shows that high overlap scores can coexist with boundary errors around 21 mm. Thin vessel–tumor interfaces are what staging studies such as Assessment of resectability of pancreatic cancer depend on. Our certification maps will show directly which on-screen pixels the base alone gets wrong.
Why not just use a neural representation like fV-SRN or a medical INR? An end-to-end implicit neural representation architecture for medical volume data reaches 40.05 dB at 97.5% compression, but on a single patient, with retraining per scan and no lossless path. Per-volume training delays the first frame, and the result is never exact. The neural-base ablation measures whether its sharper prediction shrinks residuals enough to offset the training delay.
Why not just wait for the lossless file when bandwidth is good? On a fast LAN this may well be the right answer. We will report the crossover bandwidth above which plain TCT transfer wins, rather than claim universal benefit.
Proposed Experiments
We use 16-bit CT and MRI from CHAOS, MosMedData, and Trabit, which TCT already uses, so lossless sizes are comparable. We add MSD-Liver, AMOS, and LIDC-IDRI for overlap with MIST. For each volume we script a set of views and clinically standard transfer functions (bone, vessel, soft-tissue windows) and replay them identically across methods. We emulate several bandwidth-limited links with traffic shaping on a single GPU client.
The primary metrics are certified-pixel fraction against both bytes received and wall-clock seconds, time to first frame, time to 99% certified, time to full lossless, sustained frames per second, and final bits per voxel.
The decisive checks are as follows. First, zero certified pixels may differ from the lossless rendering across all runs; any violation falsifies the method. Second, measure the gap between our bound and the true pixel error, to show the certificate is not uselessly loose. Third, the view-driven scheduler must beat the view-agnostic ablation in bytes to 99% certification; if it does not, the scheduling idea is wrong. Fourth, we must beat MIST's progressive stream at the tested bandwidths. Fifth, the final-size premium over TCT, caused by non-causal prediction, must stay small enough that a clinic would accept it.
For a clinical sanity check on CHAOS, we will report when pixels on the projected boundaries of the reference organ segmentations become certified. That is where blurring does damage, and it shows whether certification arrives early where it matters or only in easy flat regions.
References
- Practical Lossless Volumetric Medical Image Compression via Tri-plane Context Tree Learning
- A Novel Progressive Image Scanning and Reconstruction Scheme Based on Compressed Sensing and Linear Prediction
- Learned Lossless JPEG Transcoding via Joint Lossy and Residual Compression
- MMVC: Learned Multi-Mode Video Compression with Block-based Prediction Mode Selection and Density-Adaptive Entropy Coding
- A Local Clustering Algorithm for Massive Graphs and Its Application to Nearly Linear Time Graph Partitioning
- A Subquadratic Algorithm for Computing the L₁-Distance Between Two Terrains
- Towards Resource-Efficient Streaming of Large-Scale Medical Image Datasets for Deep Learning
- Fast Neural Representations for Direct Volume Rendering
- Instant Neural Representation for Interactive Volume Rendering
- CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentation
- Assessment of resectability of pancreatic cancer using novel immersive high-performance virtual reality rendering of abdominal computed tomography and magnetic resonance imaging
- An end-to-end implicit neural representation architecture for medical volume data