Very large images, such as 5000×5000 aerial scenes or drone orthomosaics of forest canopy, do not fit through a deep network on one GPU. They are therefore cut into tiles, the tiles are processed separately, and the outputs are stitched back together. Near each tile edge the network sees less of the image than it should, and it fills the missing area with padding (zeros or mirrored pixels). That is where seams, broken masks and split objects come from. The two known exact fixes each pay a different price. The first recomputes a wide overlap around every tile, which Continuous Patch Stitching for Block-wise Image Compression does after retraining the network with no padding at all. The second has neighboring devices exchange boundary strips of intermediate activations, the "halo exchange" used in physics simulations (Blocking Versus Non-Blocking Halo Exchange). Recomputation costs arithmetic that grows with network depth. Exchange costs a network message at every layer.
We propose to treat this as a wall-clock scheduling problem. For an off-the-shelf pretrained convolutional or window-attention network split across a grid of GPUs, we replace internal padding with real neighbor data, so the tiled output matches full-image inference exactly. A profiled cost model then decides, block by block, where to exchange halos and where to recompute overlap instead, and it hides each exchange behind interior computation. The claim is that this schedule is exact and needs no retraining, and that it is faster in wall-clock time than both pure recomputation and per-layer exchange. We also claim it removes the boundary failures that post-hoc blending leaves on object-level tasks such as tree-crown detection.
Core Claim
A padded network is not tied to padding. Padding stands in for pixels the tile cannot see. If every internal padding value is replaced by the true neighbor activation, and padding is kept only at the real image border, then every convolution, pointwise layer, inference-mode batch norm and windowed attention layer computes exactly what it would on the full image. This means exactness does not require the padding-free, even-kernel redesign that CPS needs. It needs the right bytes in the halo.
How to supply those bytes is a choice made separately at each layer. Suppose the network's layers are grouped into segments. Within a segment, each tile recomputes a margin equal to the segment's accumulated receptive radius $r_j$, and no messages are sent. At each segment boundary, tiles exchange a halo of width $r_j$ with up to eight 2D neighbors. The wall-clock time of a layer partition $\pi$ is approximately
$$T(\pi) = \sum_{j \in \pi} \Big[ C_j(s + 2r_j)^2 + \max\big(0,\; \alpha_j + \beta_j\, b_j(r_j) - I_j\big) \Big],$$
where $s$ is the tile side, $C_j$ is the profiled per-pixel compute time of segment $j$, $\alpha_j$ and $\beta_j$ are the per-message latency and per-byte cost, $b_j(r_j)$ is the halo size in bytes, and $I_j$ is the interior compute time available to hide the transfer. Because the objective is additive over contiguous segments, dynamic programming over layers gives the optimal partition. The two existing exact methods are the endpoints of this search: one segment is CPS-style recomputation, and one segment per layer is classic halo exchange.
Positioning Against Prior Work
The closest system is Partially Conditioned Patch Parallelism. PCPP splits a diffusion model's canvas into horizontal strips on 4–8 GPUs and sends a fraction $p$ of the neighboring activations from the previous denoising step, asynchronously. That motivates our work: it shows local neighbor exchange beats global all-gather in wall-clock time. It also constrains our design. PCPP relies on staleness across timesteps, a single feedforward pass has no timesteps, and PCPP reports seams at small $p$ and failures on rigid geometry. We make no approximation in the exact mode, we use 2D grids with corner neighbors, and we target feedforward dense prediction.
CPS supplies the exactness argument, but it requires retraining and never measures time. Bourached's halo study supplies the overlap-with-interior-work technique but concerns fluid stencils with fixed radii. Tiled Diffusion (Tiled Diffusion) replaces latent padding with neighbor slices, which is the same substitution we make, but it runs sequentially and is unprofiled.
On the wall-clock side, ADACOMM and Slow and Stale Gradients Can Win the Race show that communication frequency should be chosen by progress per second rather than per step. The G²T rule picks estimators by minimizing a cost-times-quality product. We borrow this "profile, then choose" stance and apply it to a spatial, layer-wise decision that none of these papers consider.
Research Program
We will build a drop-in wrapper for PyTorch dense-prediction models. It traces the layer graph, computes the receptive radius of each layer, rewrites internal padding into halo buffers, and assigns tiles to a 2D GPU grid. A short profiling pass measures $C_j$, $\alpha_j$, $\beta_j$ and $I_j$ on the target hardware, and dynamic programming then selects the partition. Each exchange posts non-blocking sends and receives, computes the tile interior, then computes the rim. Global operations such as image-level pooling or full global attention are reported as unsupported for exactness. We will not approximate them silently.
A second, explicitly inexact mode truncates halos to width $h < r_j$. It measures output error within a band around tile edges and chooses the fastest $h$ that keeps band error below a tolerance. This mode addresses the exactness-versus-throughput tension that the inexact randomized solver analysis and ISDNet raise. If the inexact mode gives large savings, that becomes a finding in its own right. If it gives none, then exactness is effectively free.
Baselines
The naive reference is independent tiling without overlap, using the network's native padding. This is the fastest possible embarrassingly parallel run and shows how large the boundary problem is.
The practical baseline is overlapping sliding windows with distance-weighted or spline blending, as in Benjdira et al. and Zhou et al.. We will run it on the same weights and GPUs and tune its overlap over a grid of the same size as our tolerance sweep. No reported result found on Lacuna for blending on DeepGlobe, Inria or SelvaBox, so we will measure it ourselves.
The exact-recompute baseline is CPS-style full receptive-field overlap applied to the same pretrained network. Here we keep padding only at the true image border and send no messages. This baseline isolates what communication buys us.
The per-layer halo-exchange baseline follows Bourached's non-blocking scheme at every layer. Together with exact-recompute, it brackets the planner.
The strongest single-GPU competitor on accuracy per second is ISDNet (ISDNet), which reports 74.23 mIoU at 6.90 FPS on Inria Aerial and 73.30 mIoU at 27.70 FPS on DeepGlobe. The strongest reported accuracy is BPT (Boundary-Enhanced Patch-Merging Transformer) at 77.1% mIoU on Inria. On SelvaBox the reference detector reports $RF1_{75}$ of 48.60 on the internal test set (SelvaBox).
Two ablations isolate our ingredients. The first uses fixed segment lengths instead of the planner. The second removes interior-compute overlap and waits on every exchange.
Our method beats the tiling baselines if it meets three conditions. It must match full-image outputs to floating-point tolerance wherever the full image fits. It must be strictly faster than both exact endpoints at 4 and 8 GPUs on 2D grids. And it must close the gap between blending and full-image accuracy in the boundary band. We make no claim of beating BPT's architecture. The claim is that the chosen network runs exactly, at scale, faster.
Why not just…?
Why not just blend overlapping windows? Blending averages two predictions that both lack context. It cannot recover a tree crown cut in half, and the literature documents fragmented masks and double counts under blending (Unel et al.; Bokhovkin et al.). This alternative is a real threat on region-averaged mIoU, where boundary pixels are a small fraction of the image. For that reason we report boundary-band mIoU and crown-level $RF1_{75}$ restricted to crowns that cross tile edges. If blending matches us on those metrics at lower cost, then exactness is not worth paying for, and we will report that.
Why not just recompute a wide overlap and skip communication? For shallow networks this may be the optimum, and the planner will choose it. For deep backbones the margin $r$ grows with depth, and the redundant fraction $(s+2r)^2/s^2 - 1$ approaches or exceeds one once $r$ is comparable to $s/2$. The planner comparison across backbone depths and tile sizes will show where the crossover lies.
Why not just use ISDNet or downsampling on one GPU? ISDNet's full-resolution shallow branch still has to fit in memory. Downsampling merges adjacent crowns at the centimeter scale SelvaBox needs. We test this directly on full-size SelvaBox orthomosaics and Inria tiles, recording where single-GPU methods run out of memory and what accuracy they reach on the scenes where they do fit.
Why not just stale halos as in PCPP? A single forward pass has no previous step to be stale with respect to. Staleness only becomes available for video or iterative refinement, which we leave as an extension rather than claim.
Proposed Experiments
We will run the wrapper on a convolutional segmentation network trained on DeepGlobe and Inria Aerial, and on the detector released with SelvaBox (CanopyRS/SelvaBox), at 1, 2, 4 and 8 GPUs with 1D and 2D grids.
The first decisive check is exactness. We will measure the maximum absolute difference against full-image inference on images that fit on one device. The second is the cost model: predicted $T(\pi)$ should rank partitions correctly against measured wall-clock time. The third is the Pareto frontier: we will plot wall-clock time against boundary-band error for the planner, both exact endpoints, blending, and the inexact-halo sweep. The fourth is the 2D corner cost, measuring how much the eight-neighbor exchange erodes the gains seen in 1D strips.
The proposal fails if one of two things happens. Blending might match exact outputs on boundary-crossing crowns, or the planner might never beat the better of the two endpoints by a margin larger than run-to-run timing noise.
References
- Continuous Patch Stitching for Block-Wise Image Compression
- Blocking Versus Non-Blocking Halo Exchange
- Partially Conditioned Patch Parallelism for Accelerated Diffusion Model Inference
- Tiled Diffusion
- Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD
- Slow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGD
- A Rule for Gradient Estimator Selection, with an Application to Variational Inference
- Convergence Analysis of Inexact Randomized Iterative Methods
- Streamlined Global and Local Features Combinator (SGLC) for High Resolution Image Dehazing
- Deep Image Debanding
- ISDNet: Integrating Shallow and Deep Networks for Efficient Ultra-high Resolution Segmentation
- Ultra-High Resolution Segmentation via Boundary-Enhanced Patch-Merging Transformer
- SelvaBox: A high‑resolution dataset for tropical tree crown detection
- The Power of Tiling for Small Object Detection
- Boundary Loss for Remote Sensing Imagery Semantic Segmentation