Finding Hidden Lossy Steps in Medical Imaging Code

Medical imaging models are often blamed for failing on compressed data. Chest X-rays sent through messaging apps change diagnosis, saliency maps drift off the lungs, and pathology models lose nuclear detail under JPEG. Some of that compression is visible: a hospital chooses a codec. Much of it is invisible. A Python preprocessing script quietly casts a 16-bit DICOM scan to 8 bits, saves an intermediate tile as JPEG, or drops calibration metadata. The choice may depend on a config string, a getattr on a transform name, or a fallback branch that only runs for one scanner vendor. No existing tool can read such code and say where diagnostic fidelity is lost before the model ever sees the image.

We propose a fidelity analyzer for Python medical imaging pipelines. It tracks a small abstract "fidelity state" for every image-valued variable: bit depth, whether physical units such as Hounsfield units survive, codec history and quality, and resampling. It reports each path on which that state degrades before inference. Python's dynamic features hide many of these steps, so the analyzer starts from a cheap runtime trace and extends it statically to branches the trace never ran. Where dispatch is dynamic, it asks a language model to resolve the call, and it only accepts answers the static analysis can check. The test of value is not whether warnings look plausible. It is whether a flagged lossy path predicts measured diagnostic degradation on paired compressed and uncompressed benchmarks.

Core Claim

Compression artifacts in clinical machine learning are partly a software-provenance problem, and it can be addressed before deployment. Our hypothesis is that a hybrid analysis can locate lossy operations missed by plain tracing, pattern search, and generic linters. It combines one representative execution trace, static propagation of an abstract fidelity domain, and language-model resolution of dynamic calls checked by the analyzer. The paths it flags would also explain a large share of observed prediction instability.

Neither literature currently does this. The compression literature measures harm after the fact. CheXwhatsApp defines three failure measures under roughly 90% messaging compression: prediction instability (the label changes), out-of-lung saliency, and localization instability. It mitigates them with paired training. Fischer et al. show that JPEG blurs nuclear morphology and hurts ImageNet-pretrained models when compression differs between training and inference (Abstract: Deep-learning on Lossily Compressed Pathology Images). Their later benchmark compares classical and neural codecs and proposes a task-agnostic check based on how similar pathology-model features are before and after compression (Unlocking the potential of digital pathology). All three treat the codec as a known, deliberate choice. We treat it as something to be discovered inside code.

On the program-analysis side, PYRA applies abstract interpretation (running a program over summaries of values rather than concrete values) with 56 domain types to find data-science smells such as leakage. It models no image operations, codecs, or tensors, and it lacks object-oriented and interprocedural dispatch. CALLME and ABSINT-AI show that language models help resolve dynamic JavaScript calls and choose abstractions, but only when placed inside a formal analysis. Artemis uses GPT-4o to label sources and sinks across third-party PHP APIs for taint tracking. We reuse that architectural lesson, with the model as a guide and the analyzer as the checker, and apply it to a new property: loss of image fidelity rather than security taint.

Research Program

The first component is the abstract domain. Each image-valued object carries a tuple. One field is bit depth as an interval. Another is a calibration flag recording whether a pixel-to-physical-unit mapping and its windowing parameters are still recoverable. The others are an ordered codec history with quality bounds when they can be inferred, and a resampling record. Transfer functions are written by hand for the core APIs of pydicom, PIL, OpenCV, scikit-image, NumPy, and torchvision. Examples include astype(np.uint8), cv2.imwrite with a .jpg path, PIL.Image.save(quality=...), ToPILImage, and normalization before casting. A fidelity violation is any path from a high-fidelity source (a DICOM read, a whole-slide image reader, a lossless TIFF) to a model-inference sink on which the state degrades below a policy the user declares. A double-lossy encode, which Fischer's JPEG80 recompression setting shows is harmful, is a separate violation class.

The second component is the hybrid analysis. Plain runtime tracing of the pipeline on one sample input is a strong and cheap baseline, and we build on it rather than compete with it. The trace pins down concrete types, receivers of dynamic calls, and which configuration was loaded. Static propagation then covers branches the trace did not take, such as alternative modalities, error-handling fallbacks, and config options. It also covers transforms assembled from strings, for example getattr(transforms, cfg["aug"]). For calls the analysis cannot resolve, and for third-party wrappers without transfer functions, a language model receives a slice built from interprocedural def-use chains. This addresses the intra-procedural context limitation that CALLME itself reports. The model proposes either a callee or a fidelity effect, such as "this helper re-encodes to JPEG at quality 75". A proposal is accepted only if it is consistent with the signature and data flow the analyzer sees, and repeated queries are intersected, following ABSINT-AI's finding that intersecting runs raised precision. Unresolved calls are treated conservatively as "possibly lossy" and reported separately, so the user can see both the sound result and the ranked one.

The third component links warnings to harm. For each flagged path we force execution down that path on paired data and measure the downstream effect. This turns warnings into a ranked list by expected diagnostic cost rather than a flat list of smells.

Baselines

The naive reference is no analysis at all: the pipeline as released, with degradation discovered only when a validation set happens to exercise it. Against this we report how many lossy paths reach inference unnoticed.

The strongest competitor is runtime tracing alone. We instrument the same API set with hooks that log dtype changes, codec calls, and metadata drops during execution on the repository's own test or demo inputs. It is strongest because it is precise on everything it executes and needs no language model. For fairness it receives the same inputs used to seed our hybrid, and it may also run on every config file shipped in the repository. We beat it only if we recover materially more true lossy paths located in unexecuted branches at comparable precision. On paths both methods see, we expect a tie.

The second baseline is an existing static analyzer extended with our transfer functions but no trace and no language model. This is the closest analogue of PYRA's purely static design. It isolates what the trace and the model add on dynamic dispatch. The JavaScript analogue makes the expected gap concrete: CALLME reports dynamic call-edge F1 of 0.79 against Jelly's 0.43 (CALLME). For context, PYRA reports 71% F1 on audited data-science smells (PYRA). Neither result measures fidelity loss. No reported result was found on Lacuna for detecting lossy image operations in code.

The third baseline is direct language-model review. A frontier model gets the whole repository, or retrieved files within the same token budget, and is asked to list lossy steps. Two ablations isolate the key ingredients. The first removes the language model from the hybrid, leaving trace plus static analysis with conservative treatment of unknown calls. The second replaces our interprocedural slices with CALLME-style intra-procedural windows. All language-model conditions use the same model, temperature, and query budget.

Why not just…?

Why not just trace the pipeline at runtime? This is the real threat. If most harmful lossy steps sit on the default path, tracing finds them at near-zero cost and our contribution shrinks to a convenience. The experiment below stratifies ground-truth defects by whether the repository's own inputs execute them. If the unexecuted stratum is small, we will report tracing as sufficient and narrow the claim to config-dependent and fallback branches.

Why not just train the model on compressed images, as CheXwhatsApp does? Paired training reduced prediction instability by 77–80% (CheXwhatsApp). It fixes model sensitivity to one known codec. It cannot restore Hounsfield units after an uncalibrated 8-bit cast, and it does not tell a developer that their own code introduced a second encode. The paper also reports that generic Gaussian-noise augmentation does not reproduce codec artifacts. We treat paired training as complementary and measure whether fixing flagged paths adds stability on top of it.

Why not just grep for imwrite and uint8, or ask a language model? Pattern search fails exactly where dispatch is dynamic: transforms named in YAML, wrapper functions, and methods bound at runtime. On whole-program reasoning, ABSINT-AI reports that language models asked to predict abstractions directly failed to converge on more than half of its benchmarks (ABSINT-AI). Both are run as baselines, so the size of the gap is measured rather than assumed.

Proposed Experiments

We build a defect corpus from public Python medical imaging repositories covering chest X-ray, CT, and pathology pipelines, including codebases linked to Camelyon16 such as HAG-MIL and SASHA. Two annotators label real lossy paths, and we inject seeded defects into branches the default inputs do not reach. Detection is scored by precision, recall, and F1 per violation class, stratified by executed versus unexecuted paths.

The decisive check is consequential validity. For pathology, we run each pipeline on the uncompressed RMS dataset and on PanNuke and Camelyon16, with flagged paths forced on and then repaired. We measure Fischer's feature-similarity metric, segmentation Dice, and classification AUC. For chest X-rays, we use CheXwhatsApp's paired NIH-WA and RSNA-WA data and its instability and saliency scores. The analyzer succeeds if paths it ranks as costly produce significantly larger degradation than unflagged paths, and if repairing them recovers metrics measurably beyond paired training alone. It fails if the warnings are accurate but have no effect on diagnostic quality. It also fails if runtime tracing matches its recall. Either outcome would be a useful negative result for anyone building auditing tools for clinical code.