Phone and embedded chips now include neural processing units (NPUs), fixed-function accelerators that run dense convolutions and matrix multiplies very efficiently and run almost nothing else. LiDAR 3D detection is a poor fit for them. The fast 3D detectors rely on sparse convolution, which computes only at the few occupied cells of a 3D or bird's-eye-view grid. To do that, it looks up neighbors in hash tables and gathers and scatters features at every layer. NPU compilers cannot map these operations. They split the network and send those pieces to the CPU, and the memory copies between processors erase the savings from skipping empty space. The alternative is to fill the grid densely, as PointPillars does. That keeps the model on the NPU but spends more than 95% of the work on empty cells.
We propose moving all the irregular work to a single step at the start and a single step at the end of the network. The CPU sorts occupied pillars (vertical columns of LiDAR points) into a fixed number of small square tiles, each padded with a border of empty cells. The NPU then runs an ordinary dense convolutional detector on this fixed-shape stack of tiles. Wide-range scene context comes from a coarse whole-scene branch, which is joined to the tiles through a one-hot matrix multiply, an operation every NPU supports. The dense convolutions inside each tile let features spread into nearby empty space. That spread is what sparse submanifold convolution loses and what custom accelerators add back in hardware. The hypothesis is that this recovers dense-backbone accuracy while scaling compute with occupied area, and does so on unmodified commercial NPUs.
Core Claim
Sparsity at the granularity of fixed-size tiles, with a compile-time constant tile count, is enough to get most of the compute savings of sparse convolution on a commercial NPU. It avoids per-layer graph splits and does not quantize coordinate arithmetic. It also closes the accuracy gap that fine-grained sparsity opens.
Sparse PointPillars is the evidence for that gap. It replaced the dense BEV backbone with submanifold and sparse convolutions and got 2–4× speedups on a Jetson Xavier and a robot CPU, but lost about 8.5 points of 3D AP on KITTI (Sparse PointPillars). Selectively Dilated Convolution shows the loss comes from blocked spatial spread. It lets only the top 2–4% most important pillars write into neighboring empty cells, which recovers KITTI Hard from 72.45 (submanifold) to 75.54 mAP against 75.61 for the dense model. The price is a simulated accelerator with extra dilation-check logic (Selectively Dilated Convolution). Our tiles apply the same principle, dense spread only near where things are, using nothing but primitives already in silicon.
The cost model makes the bet explicit. Let $T_g$ be the CPU time to bucket pillars into tiles, $T_s$ the time to map tile-level detections back to scene coordinates, $K$ the fixed tile count, and $t(h)$ the NPU time for one tile of side $h$ with border $r$. End-to-end latency is then
$$T \approx T_g + T_s + K\, t(h+2r) + T_{\text{global}},$$
against $T_{\text{dense}} \propto A$ for a dense pseudo-image of area $A$. $T_{\text{global}}$ is the NPU time for the coarse whole-scene branch. The gain grows with detection range and grid resolution, because $A$ grows quadratically with range while occupied area grows roughly linearly along roads.
A tile stack of $L$ stride-1 3×3 layers reproduces dense-grid outputs exactly at its centers only if $r \ge L$. That is just the receptive-field radius. Smaller borders trade exactness for speed, and measuring that trade is part of the study.
Relation to Prior Work
Four groups of prior work bear on this proposal.
Three papers motivate it by documenting the NPU problem, all on dense 2D models. A study of heterogeneous phone processors found that nearly 48% of ResNet-50 operators fell back from specialized execution on a Xiaomi 9. It also found that splitting a graph across processors can run slower than using one (Deep Learning Inference on Heterogeneous Mobile Processors). The Operator Compatibility Gap in Mobile Deep Learning reports the same fallback behavior. A neural video loop filter reached 77 FPS at 1080p on a Snapdragon 8 Gen1 NPU only by stripping itself down to plain convolutions and ReLU (Memory-speed-quality co-optimized neural loop filtering). We take that lesson as a design constraint: the NPU graph must contain only dense convolution, elementwise ops and matrix multiply.
Three systems supply sparse-execution techniques but depend on hardware that phones lack. TorchSparse pads irregular per-offset matrix multiplies into regular batches. It deliberately accepts wasted compute in exchange for regularity, which is the same bargain we make at tile scale. It is tied to CUDA and reports that INT8 helped little because scatter operations are sensitive to precision. PointAcc shows that native sparse convolution needs dedicated mapping and memory units. It reports 53× over a TPU-v3, which cannot run mapping operations efficiently. It is an ASIC, however, and does not help existing chips. SECOND built its sparse 3D detector on custom CUDA rule generation. In our design, coordinates never enter the NPU, so the INT8 problem TorchSparse reports does not arise, and we need no mapping unit.
The nearest deployment work, Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device, pruned dense PointPillars and compiled it for a Galaxy S20 GPU using reinforcement learning and Bayesian search. It reached 97 ms and 85.20% KITTI Easy car AP. It keeps the dense grid and targets a programmable GPU, and its authors list true sparse execution as future work. We target the NPU and remove empty area instead of pruning channels. The two approaches are complementary.
Two representation papers constrain the design. PointPillars supplies the pillar encoder, which is already a dense per-pillar network and runs on NPUs. Voxel or Pillar shows that pillars lose vertical detail. We therefore do not claim to beat voxel methods on accuracy, only to move pillar-level accuracy onto an NPU at lower cost. Splitting sparse inputs into dense blocks has also been used as a GPU engineering trick outside this file. What is untested, and what we contribute, is the NPU-specific version: a static tile count, a single CPU–NPU boundary, a matrix-multiply join to scene context, and training under the same tiling.
Research Program
The pipeline has four parts.
Pillar encoding. The point-to-pillar encoder from PointPillars runs unchanged as a fixed-shape dense graph on the NPU.
CPU tiling. The CPU performs one linear-time pass. It assigns each occupied pillar to a tile by integer division of its grid coordinate. It then ranks tiles by a cheap score: occupancy count, or a learned importance score computed from pooled pillar features, in the style of SD-Conv. It keeps the top $K$ tiles and writes them, with borders, into a preallocated tensor of shape $K \times C \times (h+2r) \times (h+2r)$. Scenes with fewer tiles are zero-padded. To handle the long tail of busier scenes, we compile three graphs at different $K$ values and pick among them per frame, then measure how often tiles are dropped.
NPU detector. A single NPU graph runs a dense BEV backbone and center-style detection head on every tile. In parallel it runs a coarse global branch on a 1/8-resolution occupancy-and-feature map, which the CPU builds during the same pass. The CPU also produces a $K \times G$ one-hot matrix that assigns each tile to its location among the $G$ cells of the coarse map. Multiplying by this matrix gathers global context into each tile with a supported dense operator, so there is no second graph split.
Output mapping. The CPU maps detections back to scene coordinates by adding tile offsets and runs non-maximum suppression. Training uses exactly the same tiling, including dropped tiles and zero padding, so train and test inputs match. We will measure whether training with the tiling matters, since a model trained on full dense grids and then tiled may lose accuracy near tile borders.
The analysis half of the study maps the design space. The main axes are tile side $h$, border $r$ relative to network depth, $K$ relative to scene occupancy, and learned versus occupancy-based selection. Each point is measured on the same device for latency, energy, fraction of operators running on the NPU, and AP. All measurements use INT8 through the vendor's NPU toolchain.
Baselines
Dense PointPillars on the NPU. The naive reference is PointPillars with its standard dense backbone, INT8-quantized and run through the same vendor toolchain on the same NPU. It uses our training recipe and the same epoch budget. This is the "do nothing" point: fully NPU-compatible and wasteful. The published reference is 86.10 BEV AP for KITTI Moderate car (PointPillars). To beat it, we need at least 2× lower end-to-end latency, including CPU tiling, with no more than one point of AP loss.
Dense PointPillars with a modern mobile backbone. The strongest competitor on accuracy per millisecond is PointPillars with its backbone replaced by Universal Inverted Bottleneck blocks from MobileNetV4. These blocks were designed to be Pareto-efficient across NPUs, including a 3.8 ms ImageNet model on the Pixel 8 EdgeTPU. This baseline gets the same tuning budget as our model. We will also use the same block type inside our tiles, so the only difference between the two is tiling.
Sparse execution on the CPU. Sparse PointPillars represents the sparse approach on the hardware phones do have. We run its sparse backbone on the phone CPU and its head on the NPU. Its reported accuracy cost on KITTI is roughly −5 BEV AP and −8.5 3D AP relative to dense PointPillars.
Accuracy target from custom hardware. For accuracy, the target is SD-Conv's 75.54 KITTI Hard mAP against the dense model's 75.61 (Selectively Dilated Convolution). It runs only in simulation, so we compare accuracy, not latency.
Mobile deployment reference. The published phone result is 97 ms at 85.20% KITTI Easy car AP on a Galaxy S20 GPU (Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device). The hardware differs, so we treat this as context, not a head-to-head comparison. We found no reported result on Lacuna for any LiDAR detector running on a commercial phone NPU.
Ablations. Four ablations isolate the key ingredients. First, removing the global branch tests the matrix-multiply join. Second, setting $r=0$ tests the border. Third, occupancy-only tile selection tests learned selection. Fourth, a variant where the tile backbone uses submanifold-style masking, so features cannot spread into empty cells inside a tile, tests the claim that dense within-tile spread is what recovers accuracy.
Why not just…?
Why not just run dense PointPillars with a MobileNetV4-style backbone on the NPU and accept the empty space? This is the real threat. NPUs are so fast at dense convolution that at KITTI's roughly 70 m range, empty-space waste may cost only a few milliseconds, and our CPU tiling could eat the gain. We do not argue around this. We measure it at several ranges and grid resolutions. The cost model predicts the crossover moves in our favor as range and resolution grow. If tiling fails to win even at extended range or 0.1 m pillars, the method is not worth deploying, and the paper reports that negative result.
Why not just prune and compile the dense model, as was done on the Galaxy S20? Pruning reduces channels but keeps work proportional to grid area. It also needed an expensive reinforcement-learning search tuned to a GPU (Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device). The two ideas can be combined: we will prune the tile backbone with a simple channel-pruning schedule and report whether the gains add up.
Why not run the sparse backbone on the CPU and the head on the NPU? Mobile processor studies show that splits of this kind can be slower than using a single processor and cause thermal throttling (Deep Learning Inference on Heterogeneous Mobile Processors). The approach also inherits Sparse PointPillars' accuracy loss. We run this configuration directly, as described under Baselines, and report both latency and sustained-temperature behavior over a 10-minute run.
Why not wait for sparse accelerators like PointAcc? They are not in shipping chips (PointAcc). A software method that works today on commercial NPUs also tells hardware designers how much of a dedicated mapping unit's benefit can be had without one.
Proposed Experiments
Datasets and hardware. We train and evaluate on KITTI (BEV and 3D AP, all difficulties and classes) and on nuScenes (NDS and mAP). On nuScenes the larger range makes the area argument sharper, and Voxel or Pillar gives a GPU-level accuracy ceiling of 73.8 NDS. We also run a Waymo range sweep that crops detection radius at 50, 75 and 100 m. Devices are a Snapdragon 8 Gen1 NPU through SNPE, the stack used by the loop-filter work, and a Pixel EdgeTPU. Two chips let us test whether tiling holds up across hardware fragmentation.
Measurements. For every configuration we report end-to-end latency, including pillarization, CPU tiling, NPU execution and output mapping. We also report energy per frame, the fraction of operators on the NPU (target 100% inside the model graph), tile-drop rate, and AP with and without INT8.
First decisive check. On nuScenes or Waymo at 75 m and above, the method must beat the dense MobileNetV4-block PointPillars baseline by at least 2× latency at matched AP, measured on the same NPU.
Second decisive check. The ablation with masked within-tile convolutions must lose several AP points, reproducing the submanifold gap. Removing that masking must close the gap to within about one point of dense. If both checks hold, tile-level sparsity is shown to be a practical way to run sparse LiDAR detection on commercial NPUs. If the first fails, the finding is that NPU dense throughput has made grid sparsity irrelevant at automotive ranges. That is still worth knowing.
References
- Sparse PointPillars: Maintaining and Exploiting Input Sparsity to Improve Runtime on Embedded Systems
- Selectively Dilated Convolution for Accuracy-Preserving Sparse Pillar-based Embedded 3D Object Detection
- Deep Learning Inference on Heterogeneous Mobile Processors: Potentials and Pitfalls
- Memory-speed-quality co-optimized real-timeneural loop filtering for mobile devices
- TorchSparse: Efficient Point Cloud Inference Engine
- PointAcc: Efficient Point Cloud Accelerator
- SECOND: Sparsely Embedded Convolutional Detection
- Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device
- PointPillars: Fast Encoders for Object Detection From Point Clouds.
- Voxel or Pillar: Exploring Efficient Point Cloud Representation for 3D Object Detection
- MobileNetV4: Universal Models for the Mobile Ecosystem