Abstract
Inspection of tissues using a light microscope is the primary method of diagnosing many diseases, notably cancer. Highly multiplexed tissue imaging builds on this foundation, enabling the collection of up to 60 channels of molecular information plus cell and tissue morphology using antibody staining. This provides unique insight into disease biology and promises to help with the design of patient-specific therapies. However, a substantial gap remains with respect to visualizing the resulting multivariate image data and effectively supporting pathology workflows in digital environments on screen. We, therefore, developed Scope2Screen, a scalable software system for focus+context exploration and annotation of whole-slide, high-plex, tissue images. Our approach scales to analyzing 100GB images of 10^9 or more pixels per channel, containing millions of cells. A multidisciplinary team of visualization experts, microscopists, and pathologists identified key image exploration and annotation tasks involving finding, magnifying, quantifying, and organizing ROIs in an intuitive and cohesive manner. Building on a scope2screen metaphor, we present interactive lensing techniques that operate at single-cell and tissue levels. Lenses are equipped with task-specific functionality and descriptive statistics, making it possible to analyze image features, cell types, and spatial arrangements (neighborhoods) across image channels and scales. A fast sliding-window search guides users to regions similar to those under the lens; these regions can be analyzed and considered either separately or as part of a larger image collection. A novel snapshot method enables linked lens configurations and image statistics to be saved, restored, and shared. We validate our designs with domain experts and apply Scope2Screen in two case studies involving lung and colorectal cancers to discover cancer-relevant image features.
Introduction
Traditional pathology relies heavily on Hematoxylin and Eosin (H&E) staining, which provides a two-color view of tissue structure. While effective, modern oncology increasingly requires deeper molecular insights. High-plex imaging techniques, such as CyCIF or CODEX, allow researchers to capture over 60 molecular markers simultaneously. This results in massive datasets—often exceeding 100GB—containing millions of cells across billions of pixels.
Visualizing this data presents three primary challenges: information overload (humans cannot perceive 60 colors at once), loss of context during high-magnification inspection, and workflow gaps caused by "change blindness" when switching between channel views. Scope2Screen addresses these by introducing a focus+context metaphor that bridges the gap between traditional microscopy and modern data science.
The Scope2Screen System
Scope2Screen utilizes a virtual lensing system to allow pathologists to explore high-plex data without losing the global tissue context.
Interactive Lensing and Feature Augmentation
Users can move a virtual lens over the tissue image. While the background provides the broad morphological context, the lens acts as a high-resolution focus area. This lens is not merely a magnifier; it is a data probe. As it moves, the system performs real-time spatial queries on underlying single-cell data to display histograms or radial charts of marker intensities for the specific cells currently under the lens.
One critical design choice was the rejection of "fisheye" lenses. Pathologists rely on the precise shapes of nuclei and cells for diagnosis; distortion introduced by traditional fisheye techniques was found to be counterproductive. Instead, Scope2Screen uses a "Plateau" lens that maintains a 1:1 scale in the center, only compressing the periphery.
The "Dotter" and HistoSearch
Inspired by the physical practice of "dotting" glass slides, the "Dotter" panel allows researchers to save "rich snapshots." These snapshots store the exact channel configurations, color settings, and single-cell statistics of a region, enabling instant restoration of an analytical state. To navigate these massive images, the system includes a sliding-window search (HistoSearch). By placing the lens over a region of interest, users can trigger a search to find similar regions across the entire slide based on intensity distributions.
Case Studies and Validation
The system was validated using two large-scale datasets: a 40-channel Colorectal Cancer (CRC) image (1.28 million cells) and a 38-channel Lung Adenocarcinoma image (534,000 cells).
In the lung cancer study, researchers utilized Scope2Screen to rapidly identify and characterize Tertiary Lymphoid Structures (TLS)—complex immune cell aggregates. The ability to juxtapose H&E views with specific immune markers within the lens allowed experts to validate molecular findings against known physical structures far more efficiently than manual annotation. Domain experts noted that the proximity of single-cell statistics to the visual image helped mitigate cognitive biases during cell type identification.
Conclusion
Scope2Screen provides a scalable, intuitive framework for the exploration of high-plex pathology data. By integrating real-time statistics and search capabilities directly into the visual exploration workflow, it enables pathologists to manage the complexity of multivariate images while maintaining the morphological context essential for clinical and biological discovery. Future work will look toward extending these metaphors to 3D tissue data and deeper integration with cloud-based tumor atlases.
Summary updated