Modern computer vision faces a "memory wall." As we move from standard web images to 4K video, 3D medical CT scans, and gigapixel satellite imagery, the underlying data structures—activation maps and cost volumes—quickly exceed the physical memory of even the most powerful GPUs.
While traditional data parallelism scales by processing different images on different devices, it does nothing to help when a single image is too large to fit in memory. The solution emerging across the field is spatial partitioning: dividing the visual field itself into localized regions and distributing them across a processing grid.
Breaking the Memory Wall
The core intuition behind spatial partitioning is to treat the spatial dimensions (height, width, and depth) as a distributable axis, much like the batch dimension. By splitting a large input tensor into smaller spatial sub-regions, researchers like Peter H. Jin and Kurt Keutzer demonstrated in Spatially Parallel Convolutions that memory consumption per device scales inversely with the number of processors.
This approach is particularly critical for 3D data. In High Resolution Medical Image Analysis with Spatial Partitioning, Le Hou and Noam Shazeer showed that $512^3$ CT volumes can be trained end-to-end by sharding the volume across a TPU cluster. Without this, researchers are forced to down-sample or crop their data, losing the fine-grained details essential for medical diagnosis.
The Boundary Problem and Halo Exchange
The primary challenge of spatial partitioning is that pixels are not independent. Operations like convolutions and denoising filters require a "receptive field"—a neighborhood of surrounding pixels. When an image is sliced into tiles, a processor handling the left tile lacks the pixels from the right tile needed to compute the edge of its convolution.
To solve this, the field borrows "halo regions" or "ghost zones" from high-performance computing. Before each operation, processors perform a halo exchange, swapping thin strips of boundary pixels with their neighbors.
- Raymond E. Suorsa and Banavar Sridhar pioneered this for real-time helicopter obstacle avoidance in A parallel implementation of a multisensor feature-based range-estimation method, using "Virtual Processing Regions" to ensure data locality.
- In modern rendering, Erwan Leria and Markku Mäkitalo use large rectangular tiles in Real-time Stereoscopic Image-parallel Path Tracing to minimize this synchronization overhead, ensuring that neighborhood-dependent denoising filters have the data they need without constant cross-GPU communication.
Restoring Global Context
While local tiles work for convolutions, they can cause a model to "lose the forest for the trees." Deep networks often need global context to understand large-scale structures.
In DDU-Net: A Domain Decomposition-based CNN for High-Resolution Image Segmentation on Multiple GPUs, Corné Verburg and Alexander Heinlein address this by introducing a "communication network" at the architecture's bottleneck. This allows the model to reconcile local features from different GPUs into a global representation, preventing the "seam artifacts" that often plague simple patch-based processing. Similarly, Niklas Kämper uses Domain Decomposition in Domain Decomposition Algorithms for Real-time Homogeneous Diffusion Inpainting in 4K to ensure that iterative solvers maintain global consistency across 4K image boundaries.
Beyond Pixels: Transformers and Graphs
The shift toward spatial partitioning is now moving beyond standard CNNs into newer architectures:
- Diffusion Transformers (DiTs): In PipeFusion, Jiarui Fang and Jinzhe Pan split image patches across GPUs to accelerate inference for massive models like Flux.1, reusing "stale" activations to hide communication latency.
- Graph Routing: Gabriel Díaz-Ramos and Richard Baraniuk demonstrate in MazeNet that even NP-hard graph problems can be solved by treating them as images and using spatial decomposition to process overlapping segments in parallel.
- Microcontrollers: Even on tiny devices, spatial partitioning is used to bypass memory limits. Jonathon Hare and Partha Maji show in TinyOps that breaking large tensors into height-partitioned blocks allows microcontrollers to run ImageNet-scale models that would otherwise exceed their internal SRAM.
By treating the image not as a single monolithic block, but as a collection of spatially related sub-problems, these researchers have turned the "curse of high resolution" into a massive opportunity for parallel scaling.