TL;DR
This paper tackles the significant I/O bottleneck in digital pathology by implementing and evaluating the ADIOS2 data management framework for whole slide images (WSIs). They demonstrate that ADIOS2 can significantly speed up data access in CPU-based analysis and achieve performance comparable to specialized GPU I/O solutions in GPU-accelerated pipelines, making large-scale WSI analysis more efficient.
What Problem Does This Paper Address?
Digital pathology, with its massive whole slide images (WSIs) that can be gigapixels in size, presents a substantial data management challenge. Analyzing these images, especially when broken down into smaller patches for processing (a common approach for algorithms), leads to a huge number of input/output (I/O) operations. This I/O bottleneck often becomes the limiting factor, slowing down the entire analysis pipeline, even when powerful CPUs or GPUs are used for computation. The paper aims to alleviate this by improving how data is accessed and managed for WSI analysis.
Methods
The core of this work is the implementation and evaluation of the Adaptable IO System version 2 (ADIOS2), a framework designed for high-performance scientific data management. ADIOS2 uses the Message Passing Interface (MPI) for parallel I/O and stores data in a self-describing binary-packed (.bp) format. The authors developed a WSI-centric pipeline using ADIOS2 and evaluated it in two scenarios:
- CPU Scenario: This involved reading WSI data (converted to .bp format by ADIOS2) and extracting 512x512 patches for analysis. They compared ADIOS2's performance against traditional methods of loading entire WSIs into memory or loading individual patch files.
- GPU Scenario: Here, they integrated ADIOS2 into an existing GPU-accelerated deep learning segmentation pipeline called "Accelerated+ Omni-seg." They compared three approaches: the original pipeline, a multi-process ADIOS2 framework, and NVIDIA's Magnum IO GPUDirect Storage (GDS).
Datasets
The study used 100 kidney Whole Slide Images (WSIs) from the Kidney Tissue Atlas (KPMP) Datasets. These WSIs were acquired at a high resolution (0.25 Micron) and stained with various techniques (H&E, PAS, Silver, Toluidine Blue, and Trichrome) to ensure a comprehensive evaluation across different tissue structures and staining types.
Key Insights and Ideas
- ADIOS2 as a General-Purpose I/O Accelerator: The paper highlights ADIOS2's versatility, showing its effectiveness not just in traditional HPC but also in specialized domains like digital pathology.
- Parallelism is Key: The significant speedups observed, especially in the CPU scenario, underscore the power of parallelizing I/O operations when dealing with large datasets like WSIs.
- Bridging the Gap for GPU: While GPUs excel at computation, efficient data movement remains crucial. ADIOS2 offers a way to improve this, even when specialized GPU I/O solutions exist.
- Early Adoption in Digital Pathology: The authors note that this is one of the first applications of ADIOS2 in digital pathology, suggesting a promising avenue for future research and adoption.
Key Results
- CPU Scenario: ADIOS2 achieved an impressive two-fold speed-up in I/O operations compared to a brute-force approach (loading WSI-wise numpy files). It was also significantly faster than processing patch-wise numpy files.
- GPU Scenario: ADIOS2's performance was on par with the cutting-edge NVIDIA Magnum IO GPUDirect Storage (GDS) framework. While GDS was slightly faster, ADIOS2 provided comparable performance, demonstrating its viability for GPU-accelerated WSI analysis.
- The median total processing time for ADIOS2 was the lowest in the CPU scenario and comparable to the original approach in the GPU scenario.
Limitations and Future Work
- GPU Scenario Nuances: While ADIOS2 performed comparably to GDS in the GPU scenario, the paper notes that the I/O time reduction was "not as pronounced" compared to the CPU scenario. This is likely because the GPU itself already reduces substantial processing time, making I/O less of a bottleneck relative to computation.
- Future Work: The paper doesn't explicitly detail future work, but the successful application of ADIOS2 suggests potential for further optimization and integration into more complex digital pathology workflows. The availability of the source code also encourages community adoption and development.
Summary updated