Analog Encoders, Exact Priors: Splitting Joint Point Cloud Codecs Across In-Memory Hardware

A learned compressor for 3D point clouds, meaning sets of 3D points that each carry extra values such as color or LiDAR reflectance, has two very different kinds of computation inside it. The first is a large network that turns the input into a compact code. The second is a smaller network that predicts how likely each symbol of that code is, and an arithmetic coder turns those predictions into bits. On edge sensors such as cars, robots, and headsets, the energy cost of running such a codec is set mostly by moving weights and activations between memory and compute, not by the arithmetic. This is the von Neumann bottleneck. In-memory computing removes most of that traffic by storing weights as conductances in an analog array and multiplying in place. Analog arrays are noisy, though. A probability model that gives slightly different numbers at the encoder and the decoder does not just lose a little accuracy. It makes the bitstream undecodable.

We propose a codec whose structure follows this asymmetry. Any computation upstream of a quantizer runs on analog in-memory arrays. Any computation that produces a probability runs in deterministic integer logic. Because the analog noise ends at a rounding step, it can only make the encoder somewhat worse. It can never break decoding. We then ask how codec capacity should be redistributed when memory movement is the dominant cost. Our hypothesis is that the right design is a heavy analog analysis transform paired with a deliberately tiny exact prior, with one shared trunk for geometry and all attribute channels. A single programmed array then serves every attribute type.

Core Claim

Joint geometry-attribute codecs can move most of their encoder computation onto analog in-memory hardware with decodability guaranteed by construction, under one rule: analog may compute features but never probabilities. We further claim that under in-memory energy accounting, the best rate-distortion-energy trade-off comes from shrinking the context and hyperprior networks and growing the analysis transform. This is the opposite of the trend in recent codecs, which put much of their capacity into context modeling.

The claim has a sharp corollary that we will test. How much of a codec's computation is eligible for analog execution depends on its design family. Lossless-occupancy geometry coders spend nearly all of their neural computation on probability prediction. Lossy transform coders spend most of it on the analysis transform. Examples of the first family are RENO's one-shot sparse occupancy codes (RENO) and RangeCM's range-image context models (Low-Latency Neural LiDAR Compression with 2D Context Models). An example of the second is MEGA-PCC's joint autoencoder (MEGA-PCC). The value of in-memory hardware for point cloud compression is therefore a property of codec architecture, not only of the hardware.

Where This Sits

Several papers motivate the energy argument. The Sze et al. survey (Hardware for machine learning) establishes that data movement dominates machine-learning energy. It names in-memory computing as a remedy but studies 2D image networks only. PointAcc (PointAcc) shows that gathering and scattering sparse features takes over half the runtime of point cloud networks on GPUs, and fixes this with a digital accelerator. PointAcc targets perception tasks, not codecs, and has no analog substrate.

Other papers supply the method and also constrain it. Analog Foundation Models (Analog Foundation Models) and Spoon et al. (Toward Software-Equivalent Accuracy on Transformer-Based DNNs With Analog Memory Devices) give the hardware-aware training recipe: noise injection, weight clipping, and drift compensation. We reuse that recipe for the analysis transform. Spoon et al. also report that once matrix products go analog, the remaining digital operations become the energy bottleneck. That finding is exactly why our digital prior must be small.

MemTorch (MemTorch) adds a constraint: accuracy falls off a cliff when converter resolution drops below 4–5 bits. This bounds how coarse our array readout can be.

On the compression side, the codecs we build on all run on digital GPUs and none considers noisy compute. RangeCM shares one backbone between range and intensity. MEGA-PCC unifies geometry and color with Mamba blocks and a dual decoder. RENO handles geometry only and explicitly leaves attributes open. None of the three asks which parts of the network may tolerate non-determinism.

To our knowledge, no paper in this space has put a compression codec on in-memory hardware. The obvious version would be "simulate RangeCM on a crossbar" (Gap 1 in our notes). We differ from that version in three ways. We treat encoder–decoder determinism as the governing constraint. We derive the partition rule from it. And we change the codec architecture to fit the hardware's cost structure, rather than porting an existing codec.

Research Program

The system has three parts.

The first is an encoder-side analysis transform executed on simulated analog arrays. It consists of the main analysis network and the hyper-analysis network. Both feed a rounding quantizer, and the quantized code is digital from that point on.

The second is a deterministic integer prior, run identically at the encoder and at the receiver's decoder. It covers the hyper-synthesis network and any spatial or channel context model.

The third is a digital synthesis transform at the receiver. We assume the deployment pattern is sensor-to-server, so decoding happens on digital hardware.

The encoder computes rate with the same exact prior the decoder uses. Analog noise therefore shows up only as a slightly suboptimal code, plus a measurable rate increase over time as device conductances drift.

For training, we inject a hardware-calibrated noise model into the analysis path only. The noise models come from AIHWKit (Using the IBM Analog In-Memory Hardware Acceleration Kit) and MemTorch, covering programming noise, read noise, converter quantization, and drift. The synthesis transform and the prior see noisy codes during training and learn to be robust to them. We follow MEGA-PCC's two-stage schedule: attributes are first trained on ground-truth geometry, then the whole codec is trained jointly.

The attribute-agnostic element is a single shared trunk. Geometry occupancy features and an arbitrary stack of attribute channels enter it together, and small per-attribute input and output heads handle channel-count differences. On in-memory hardware, a decoupled design needs either two programmed arrays or reprogramming of non-volatile cells, and writes are slow and wear the device. A shared trunk needs one array; adding an attribute costs only activations.

The architectural study sweeps how capacity is split between the analysis transform and the prior at fixed total parameter count. For each split we trace rate, distortion, and estimated energy.

Energy is estimated with a parametric accounting model. It counts analog array operations, converter conversions, digital integer multiply-accumulates, and on-chip and off-chip memory accesses. The file gives no absolute per-operation energy figures we can rely on. We therefore do not fix them. Instead we sweep the relative cost of DRAM access to multiply-accumulate across a wide range. We report only conclusions that hold across that range, and say explicitly where they flip.

Mamba's selective scan is a recurrence rather than a matrix product, so it does not map naturally to crossbars. Our main joint codec therefore uses convolutional blocks. MEGA-PCC itself is kept as a full-precision digital reference.

Baselines

The naive reference point has two parts. One is G-PCCv23 run on its standard configuration. The other is the decoupled pipeline: a separate geometry codec followed by an attribute codec. These anchor the rate axis. MEGA-PCC reports average BD-rate savings over G-PCCv23 on 8iVFB and Owlii of 80.4% in D1-PSNR, 43.6% in Y-PSNR, and 49.6% in 1-PCQM (MEGA-PCC). RENO reports 12.25% bit-rate savings over G-PCCv23 for LiDAR geometry (RENO).

The strongest existing compression method on LiDAR is RangeCM. It reports a 14.9% BD-rate improvement over RIDDLE on the Waymo Open Dataset and roughly 0.1 s per frame on a desktop GPU (Low-Latency Neural LiDAR Compression with 2D Context Models). We run it from the public code (RangeCM codebase) at full precision, retrained by us on the same Waymo and KITTI splits. It serves as the rate-distortion ceiling for a digital implementation. On colored point clouds, the corresponding ceiling is MEGA-PCC, reimplemented and trained on ScanNet and RWTT.

The strongest competitor on the actual question of energy is the same codec quantized to INT8 and INT4 on a digital edge accelerator. We give it PointAcc-style on-chip caching and layer fusion in our energy model, because PointAcc reports a 3.5–6.3× reduction in DRAM access from these techniques. We found no reported result on Lacuna for quantized RangeCM or quantized MEGA-PCC. Both are run by us with the same training data and the same quantization-aware fine-tuning budget we give our hardware-aware analog training.

Our method "beats" this baseline only if it sits on a better rate-distortion-energy front. Concretely, it must use lower estimated encoder energy at a BD-rate loss of a few percent or less against the full-precision codec. This has to hold across most of the swept memory-cost range, not at a single favorable setting.

Two ablations isolate the key ingredients. The first is a fully analog codec, with the prior also on arrays, trained with the same noise injection. It tests whether the determinism rule is needed at all. The second is the partitioned codec with the original, unrebalanced capacity split. It tests whether shifting capacity from prior to analysis transform adds anything beyond the partition itself. A third, smaller ablation compares the shared trunk against two separate encoders at matched parameters. It tests the attribute-agnostic claim.

Why not just…?

Why not just quantize the whole codec to INT4 and run it on a digital accelerator? This is the real threat. Spoon et al. report a 91% energy reduction for INT6 digital periphery, which shows how far digital precision reduction can go. Our design also runs the prior digitally, so if the analysis transform accounts for only a small share of encoder energy, the analog gain shrinks to almost nothing. Analog Foundation Models reports a 3.6× efficiency gain over idealized INT8 for dense language models. Point cloud codecs are not dense language models, so that figure does not transfer. The capacity-split sweep settles the question directly by measuring the analog-eligible fraction of energy. If the digital INT4 front dominates, we will report it. That would be a useful negative result for edge codec design.

Why not just run everything analog with hardware-aware training, as was done for language models? The training recipe makes classification and generation robust because small output errors cost small accuracy. Arithmetic decoding has no such tolerance: one probability that differs between encoder and decoder corrupts every symbol after it. The fully analog ablation measures the frame decode-failure rate under realistic noise and under simulated drift over time. We expect that rate to be far from zero even after hardware-aware training. If it turns out near zero after coarse rounding of the predicted probabilities, our partition rule is overly conservative, and we will say so.

Why not just project to 2D range images and use a GPU? RangeCM shows this is fast. But its 0.1 s per frame on a desktop GPU uses the whole budget of a 10 Hz sensor at desktop power. The projection also does not apply to unstructured colored clouds such as 8iVFB. We include RangeCM in the study precisely because it is the strongest case for "context-heavy codecs are fine." Measuring how little of its computation is eligible for analog execution is part of our claim, not something we need to argue around.

Why not just a PointAcc-style digital accelerator? PointAcc's caching addresses gather and scatter traffic for perception networks. It does not address weight traffic in large analysis transforms. We fold its caching gains into the digital baseline's energy model, so any remaining advantage is attributable to keeping weights stationary in the arrays.

Proposed Experiments

We use two settings. The LiDAR setting codes geometry plus reflectance on the Waymo Open Dataset and KITTI, building on RangeCM. The colored-cloud setting codes geometry plus RGB on 8iVFB and Owlii, with training on ScanNet and RWTT. Analog behavior is simulated with AIHWKit's phase-change-memory models, cross-checked with MemTorch's resistive-memory models. Converter resolution is swept from 8 bits down to 4 bits.

We report three groups of metrics. Compression quality is BD-rate against the full-precision codec and G-PCCv23, measured in D1-PSNR, Y-PSNR, and 1-PCQM for colored clouds, and in range and intensity error for LiDAR. Hardware behavior is the frame decode-failure rate, the rate increase after simulated one-day, one-week, and one-month drift, and the fraction of encoder computation and energy executed in analog arrays. Task impact on SemanticKITTI-style LiDAR is detection accuracy on decoded scans, following RENO's CenterPoint protocol; RENO reports 65.64 mAP from compressed data against 65.97 mAP from raw data.

Three checks decide whether the claim holds. First, the partitioned codec must show zero decode failures while the fully analog ablation does not. Second, on transform-heavy codecs, the rebalanced codec must dominate the INT4 digital baseline on the rate-distortion-energy front across most of the swept memory-cost range. On context-heavy codecs, we predict it will not, and the difference between the two families is itself the finding. Third, drift should raise rate gradually without ever causing a decoding error. That property is the concrete deployment advantage of putting the boundary between analog and digital exactly at the quantizer.