For decades, the standard workflow in scientific research followed a simple "download and analyze" pattern: a researcher would identify a dataset, transfer it to their local workstation or institutional cluster, and run their analysis scripts. However, as scientific disciplines like genomics, climate science, and seismology enter the "petascale era," this model has reached a breaking point.
When datasets reach petabytes in size, they develop what Siyuan Lu and Conrad M Albrecht describe as data gravity. The sheer volume of information makes it physically and financially impossible to move. Transferring a single petabyte over a standard high-speed network can take months, effectively locking the data away from the researchers who need it. To solve this, the research community is shifting toward a new paradigm: instead of moving the data to the code, we must move the code to the data.
The Data Commons Model
The primary architectural response to data gravity is the Data Commons. As defined by Robert L. Grossman in Ten lessons for data sharing with a data commons, a data commons is a cloud-based ecosystem that co-locates massive datasets with the computing power and software tools required to analyze them.
In this model, the "science gateway" serves as a unified entry point. Rather than providing a download link, platforms like the Earth System Grid Center for Enabling Technologies (ESG-CET), described by D. N. Williams and Rachana Ananthakrishnan, provide server-side analysis capabilities. Researchers submit their queries or algorithms to the gateway, which executes the computation directly on the storage nodes. This minimizes data movement, as only the high-value results—often orders of magnitude smaller than the raw data—are sent back to the researcher.
Cloud-Native Analysis Facilities
Transitioning to a "compute-to-data" model requires a fundamental rethink of how scientific data is stored and accessed. Traditional POSIX-compliant file systems, which treat data as a hierarchy of files and folders, struggle to scale to the exabyte level. D. Ciangottini and colleagues argue in the Analysis Facilities for the HL-LHC White Paper that future infrastructure must transition toward cloud-native object storage.
Object storage allows for massive parallelism. In seismology, Yiyu Ni and Marine Denolle demonstrated this by processing 1.3 petabytes of global seismic data in just three days using AWS. By leveraging containerization (like Docker or Singularity), researchers can package their entire computational environment—including specific library versions and dependencies—and ship it to the cloud. This ensures that the code runs exactly the same way on a remote storage node as it did on the researcher's laptop, solving the "installation hell" described by Wei Ouyang.
From Data-to-Algorithms to Algorithms-to-Data
This shift is perhaps most visible in the concept of Evaluation-as-a-Service (EaaS). In traditional machine learning or information retrieval research, a central body releases a test dataset, and researchers download it to train their models. Allan Hanbury and Frank Hopfgartner advocate for reversing this: the Algorithms-to-Data approach.
In an EaaS framework, the data remains centralized and private. Researchers submit their algorithms (often as Virtual Machines or containers) to the data provider. The provider runs the algorithm against the hidden data and returns only the performance metrics. This not only handles the scale of massive web crawls or sensitive medical records but also ensures perfect reproducibility, as the exact code used for the evaluation is preserved by the platform.
Efficiency Through Specialized Indexing
Moving compute to the storage layer also allows for optimizations that are impossible in a download-heavy world. For example, the IBM PAIRS Geoscope, detailed by Conrad M Albrecht, indexes geospatial data at the pixel level across multiple layers (satellite imagery, weather, land use).
Because the computation happens where the data lives, the system can perform "scanning queries" that read the data once and pass it through a pipeline of concurrent analytics. Maria T. Patterson describes a similar mechanism in the Matsu Wheel, which batches multiple researcher requests into a single pass over the data. This turns a linear I/O bottleneck into a constant-time operation, enabling large-scale reanalysis of historical archives that would otherwise be too "heavy" to touch.
By treating data as a service rather than a set of files, these architectures—from C3Grid in climate science to Daisy in X-ray physics—are democratizing access to petascale science, allowing individual researchers to perform global-scale analysis without needing a supercomputer in their basement.