Accelerated Decoding of Centroid Positional Encoding for Instance Segmentation

Carmelo Scribano, Filippo Muzzini, Nedyalko Prisadnikov, Mohammad Mahdi, Yuqian Fu, Giorgia Franchini, Danda Pani Paudel, Marko Bertogna, Luc Van GoolarXiv 2026 · arXiv

TL;DR

While deep learning researchers frequently focus on optimizing neural network backbones, the "decoding" stage—turning raw network outputs into usable masks or boxes—often remains a massive computational bottleneck. This paper introduces a highly optimized CUDA implementation for a state-of-the-art instance segmentation method, eliminating memory contention and achieving up to a 55% end-to-end speedup on edge devices like the NVIDIA Jetson.

What Problem Does This Paper Address?

Bottom-up instance segmentation models offer a elegant path toward real-time processing because they predict dense, pixel-level embeddings to group pixels into objects without relying on rigid bounding box proposals. A prominent recent approach uses sinusoidal centroid encodings, where each pixel regresses a positional embedding pointing to its object's center.

However, translating these dense mathematical embeddings back into actual instance masks (the decoding phase) is computationally brutal. Traditionally, this is handled by slow CPU routines or naive GPU kernels plagued by global memory contention (atomicAdd bottlenecks) and poor memory reuse. On resource-constrained edge devices, this post-processing step can completely wipe out the latency gains achieved by a fast backbone. The authors set out to bridge this gap between theoretical representation and practical deployment.

Methods

The authors target the decoding pipeline of the sinusoidal centroid encoding method, which traditionally consists of three steps: accumulating a coarse voting histogram to find candidate object centroids, performing a local-maxima search to pinpoint them, and aggregating pixels into masks based on their affinity to these centroids.

Overview of the decoding process

To accelerate this, they completely overhauled the pipeline using custom CUDA kernels:

  1. Mathematical Decomposition for Histogram Accumulation: In naive implementations, every thread computes a pixel-bin pair and uses slow atomicAdd operations to update a global histogram, causing severe serialization. The authors elegantly factorized the exponential voting weight formula, transforming the histogram accumulation into a standard matrix multiplication ($H = E_x^T E_y$). This allowed them to offload the heaviest computation to the highly optimized cuBLAS library, completely bypassing atomic contention.
  2. Shared Memory Caching for Mask Aggregation: During the final pixel-to-mask assignment, they structured the kernel to launch one thread per pixel and loaded the pixel's affinity values into fast, on-chip GPU shared memory. This avoids repeatedly reading the same vectors from global memory when evaluating multiple candidate centroids.

Datasets

The underlying instance segmentation model was evaluated using the standard COCO dataset benchmark, utilizing a powerful DINOv2 backbone. Because this paper focuses entirely on accelerating the post-processing stage of an existing encoding scheme, no new datasets or benchmarks were introduced.

Key Insights and Ideas

  • Post-Processing is a First-Class Citizen: Model efficiency isn't just about reducing FLOPs in the backbone. For real-time systems, the output representation and its decoding algorithm must be co-designed.
  • Matrix Multiplication as a Cure for Contention: The core "aha" moment of the paper is realizing that a complex, contention-heavy spatial voting step can be mathematically refactored into a standard GEMM (General Matrix Multiply) operation, unlocking decades of hardware-level optimization via cuBLAS.
  • The 32×32 Sweet Spot: Through rigorous testing, the authors found that increasing the histogram bin resolution beyond 32×32 yields diminishing returns for Panoptic Quality (PQ) but scales latency drastically. This establishes a clear Pareto frontier for real-time edge applications.

Key Results

The optimized decoder was implemented as a TensorRT plugin and profiled on an NVIDIA Jetson Orin Nano:

  • End-to-End Speedups: The proposed decoder consistently reduced end-to-end latency, achieving a speedup of up to 55% in INT8 precision mode (at a 280×280 input resolution) compared to the PyTorch baseline.
  • Kernel-Level Efficiency: In isolation, the optimized Mask Aggregation kernel delivered an ~6-fold speedup over the naive GPU implementation.
  • Timing Stability: By utilizing static memory pre-allocation and eliminating global memory contention, the implementation demonstrated exceptional timing stability (low standard deviation in execution time), which is vital for safety-critical edge AI applications.
Latency analysis across implementations and resolutions

Limitations and Future Work

Despite drastically curtailing raw GPU compute time, memory operations remain the primary bottleneck, now accounting for roughly 33% of the total decoder inference time. The authors acknowledge that future research must shift away from framework-level memory handling toward custom, low-level memory orchestration to further minimize latency on edge hardware.

Summary updated

Abstract

Beyond model inference, the decoding stage, which converts raw network outputs into task-level representations, constitutes a significant portion of the execution cost. Despite its practical impact, prediction decoding has received comparatively little attention and is often implemented using generic CPU routines or inefficient GPU kernels, limiting the benefits of advances in model efficiency. In this work, we investigate the decoding overhead associated with a recent sinusoidal centroid encoding for Instance Segmentation, in which each pixel regresses a positional embedding of its instance centroid. This approach allows flexible segmentation without predefined proposals, but extracting instance masks from dense embeddings incurs a high computational cost. We present an optimized CUDA-based implementation of the decoding algorithm tailored to this encoding, explicitly addressing challenges related to parallelization, synchronization, and memory access on modern GPUs. Our solution significantly reduces decoding overhead and improves End-to-End inference latency, outperforming both CPU-based approaches and naive GPU implementations. The results demonstrate that efficient decoding is essential to fully exploit the advantages of advanced output representations and highlight the importance of jointly designing encoding schemes and their decoding algorithms for real-time computer vision systems.
BibTeX
@misc{scribano2026accelerated,
  author = {Carmelo Scribano and Filippo Muzzini and Nedyalko Prisadnikov and Mohammad Mahdi and Yuqian Fu and Giorgia Franchini and Danda Pani Paudel and Marko Bertogna and Luc Van Gool},
  title = {{Accelerated Decoding of Centroid Positional Encoding for Instance Segmentation}},
  year = {2026},
  eprint = {2609.16874},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.16874},
}
Figure