PathoJev: Efficient Pathology Decisions from Shared Visual Context

Chengyang Zhang, Wenchuan Zhang, Bo Li, Xinyu Liu, Jiaming Yang, Mengran Li, Jie Chen, Wei Ju, Yuhao Yi, Hong Bu, Jiancheng LvarXiv 2026 · arXiv

TL;DR

PathoJev is a specialized pathology model that makes multiple diagnostic decisions from a single whole-slide image (WSI) encoding by directly predicting answer probabilities. By moving away from slow text generation to a "System One" direct-decision paradigm, it achieves state-of-the-art accuracy while running up to 9.1x faster than existing generative models.

What Problem Does This Paper Address?

In clinical pathology, a single tissue slide requires many distinct judgments: identifying the tumor type, assessing its histologic grade, and checking for signs of invasion. Current pathology AI models are mostly "Vision-Language Models" (VLMs) that act like chatbots. To answer three questions about one slide, they typically re-read the massive visual data of the whole-slide image three times and generate text responses word-by-word.

This process is incredibly inefficient. Whole-slide images are enormous, and re-processing them for every question creates a massive computational bottleneck. Furthermore, parsing text answers (e.g., "The grade is 2") into usable data adds another layer of latency and potential error.

Methods

The authors introduce PathoJev, which shifts from a generative "System Two" (slow, deliberative) approach to a "System One" (fast, direct) paradigm.

  • Direct Probabilistic Readout: Instead of "writing" an answer, PathoJev uses a specialized decision head to directly output probabilities for predefined choices (e.g., Grade 1, 2, or 3). This is faster and provides a built-in measure of confidence.
  • Shared Visual Context: The model encodes a WSI once into a "visual prefix." Multiple independent questions then branch off this single prefix and are evaluated in a single batch.
  • Hybrid Backbone Architecture: PathoJev uses a hybrid model (based on the Qwen family) that combines recurrent layers with standard attention layers. The recurrent states summarize the long visual context into a fixed size, which prevents the computational cost from exploding as more questions are added or as the slide context grows longer.
Overview of PathoJev's architecture: WSI features are encoded once into a shared prefix, allowing multiple independent questions to be processed efficiently in parallel.

Datasets

The models (9B and 27B parameters) were trained on a massive mixture of pathology data:

  • Whole-Slide Data: SlideInstruct, WSI-Bench, and WSI-VQA.
  • ROI Data: PathVQA and the pathology subset of PMC-VQA.
  • CF-Path (New): The authors introduced a new training set of 126,467 binary statements derived from TCGA pathology reports to teach the model diverse diagnostic concepts.

Evaluation was performed across three whole-slide benchmarks (including SlideBench-TCGA) and five region-of-interest (ROI) benchmarks.

Key Insights and Ideas

  • Efficiency through Reuse: By caching the visual prefix and using recurrent states, the "marginal cost" of answering an additional question becomes negligible. This makes complex, multi-question diagnostic checklists feasible in real-time.
  • Structure Beats Generation: For structured diagnostic tasks, direct probability prediction is not just faster—it's more accurate. It avoids the "hallucinations" and formatting errors common in generative models.
  • Unified Framework: PathoJev successfully uses the same probabilistic interface for both global whole-slide assessments and local region-of-interest (ROI) judgments, creating a consistent tool for pathologists.

Key Results

  • Superior Accuracy: On whole-slide benchmarks, PathoJev-9B and 27B achieved mean accuracies of 77.2% and 78.1%, respectively, significantly outperforming previous state-of-the-art models like SlideChat (71.8%).
  • Massive Speedups: PathoJev-9B is 9.1x faster than SlideChat on the WSI-Bench. When handling a workload of 64 questions on a single slide, PathoJev is up to 11.2x faster than models that don't share visual context.
  • Well-Calibrated Confidence: The model's probabilities are reliable; for instance, by only keeping the top 80% most confident answers, accuracy on WSI-Bench jumps from 90.4% to 97.0%.
Figure 1. PathoJev: shared visual context for multiple pathology decisions. (a,b) Autoregressive answers versus direct probabilities from isolated questions sharing slide context. Re-reading context includes accessing cached prefix keys and values. (c,d) Whole-slide and ROI accuracy–latency comparisons. Accuracy averages each model’s available results across three WSI or five ROI benchmarks. Latency is GPU forward time per question on WSI-Bench and PathMMU-tiny, respectively. Zero-shot WSI baselines use thumbnails.

Limitations and Future Work

  • Fixed Answer Spaces: PathoJev currently requires predefined answer candidates (multiple-choice or binary). It cannot yet handle open-ended descriptive questions or continuous measurements (like "What is the exact tumor diameter?").
  • Workflow Orchestration: While PathoJev is an efficient "engine," a higher-level system is needed to decide which questions to ask based on a specific clinical goal.
  • Multi-Slide Integration: Future versions could be extended to aggregate evidence across multiple slides, different stains (e.g., H&E and IHC), and various magnifications for a single patient case.

Summary updated

Abstract

Pathology workflows require multiple visual judgments about the same slide, from identifying tumor type to assessing grade and invasion. Generating and parsing a separate textual response for each structured question adds latency between visual evidence and usable decisions. Building on the System One paradigm, we introduce PathoJev, a pathology model that directly predicts answer probabilities for whole-slide and region-level assessment. PathoJev encodes each slide once and evaluates independent question suffixes in batches over the shared visual context. Reusing the hybrid backbone's recurrent states further limits the computation that grows with slide-context length for each additional question. Across three whole-slide benchmarks, the 9B and 27B models achieve mean accuracies of 77.2% and 78.1%, exceeding SlideChat by 5.4 and 6.3 percentage points. Both scales also maintain competitive accuracy on five ROI benchmarks. On WSI-Bench, the 9B model runs 9.1x faster than SlideChat. Shared context accelerates eight-question workloads by 6.9x and 5.9x over separate prefix encoding, rising to 11.2x and 8.9x for 64 questions. These results provide a reusable probabilistic interface for combining multiple visual judgments in whole-slide and local tissue assessment. Code: https://github.com/ChyaZhang/PathoJev.
BibTeX
@misc{zhang2026pathojev,
  author = {Chengyang Zhang and Wenchuan Zhang and Bo Li and Xinyu Liu and Jiaming Yang and Mengran Li and Jie Chen and Wei Ju and Yuhao Yi and Hong Bu and Jiancheng Lv},
  title = {{PathoJev: Efficient Pathology Decisions from Shared Visual Context}},
  year = {2026},
  eprint = {2610.40752},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.40752},
}
Figure