Space-Filling Curve Serialization for 3D Deep Learning

236 concepts119 papers1 proposal

The Dimensionality Dilemma in 3D Deep Learning

Deep learning architectures thrive on structure. Standard convolutional networks expect rigid 2D pixel grids, while Transformers and State Space Models (SSMs) like Mamba operate on 1D sequences of tokens.

Three-dimensional spatial data—such as point clouds, voxels, and mesh surfaces—defies both formats. Point clouds are inherently unordered, unstructured sets of coordinates. Voxels provide structure but scale cubically ($O(N^3)$), leading to massive computational overhead.

To leverage the power of sequence models, we must flatten 3D data into a 1D sequence. However, naive serialization strategies introduce a severe penalty. If we flatten a 3D grid using standard row-major order (scanning line by line, layer by layer), points that are physically adjacent in 3D space often end up separated by massive gaps in the 1D sequence.

Naive Scan (Row-Major):
(0,0,0) -> (1,0,0) -> (2,0,0) ... [End of row] ... -> (0,1,0)

For sequence models, this spatial disruption is catastrophic. In an autoregressive Transformer, a model must look across a massive context window to find spatial neighbors. In State Space Models, which rely on a continuous hidden state to pass information forward, this separation causes "locality forgetting." The model's finite hidden memory is forced to retain information over hundreds of irrelevant tokens before encountering a spatial neighbor, degrading its ability to capture local geometric structures.

Preserving Locality via Space-Filling Curves

To bridge the gap between 3D spatial topology and 1D sequential architectures, researchers use space-filling curves (SFCs), most notably the Hilbert curve and the Morton (Z-order) curve.

Morton (Z-order) Curve:       Hilbert Curve:
   ┌─┐   ┌─┐                     ┌─┐   ┌─┐
   │ │   │ │                     │ └───┘ │
 ┌─┘ └───┘ └─┐                   └─┐   ┌─┘
 │           │                     │   │

An SFC is a continuous fractal path that weaves through a multi-dimensional space, visiting every routing element exactly once. The defining mathematical property of these curves is locality preservation: they ensure that elements proximal in the high-dimensional space remain proximal within the linearized 1D sequence.

By traversing 3D space along an SFC, we can serialize unstructured or volumetric data into a token stream where local neighborhoods are preserved. Dingkang Liang et al. demonstrate this in PointMamba: A Simple State Space Model for Point Cloud Analysis, where mapping point clouds to a 1D sequence via Hilbert curves provides the spatial consistency necessary for causal modeling. Similarly, Guowen Zhang et al. utilize Hilbert curves in Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detection to serialize 3D voxels, allowing an SSM backbone to achieve state-of-the-art detection accuracy with linear computational complexity.

Why SFC Serialization Empowers Sequence Models

SFC serialization works because it aligns the physical layout of data with the inductive biases of sequence architectures.

1. Smooth State Transitions in SSMs

In a State Space Model, the hidden state $h_t$ updates based on the current token $x_t$ and the previous state $h_{t-1}$. If the sequence follows a locality-preserving curve, $x_t$ and $x_{t-1}$ are highly likely to be spatial neighbors. The hidden state can evolve smoothly along the geometric manifold, capturing local structural features without needing to maintain long-range memory buffers. This is why the authors of RainMamba: Enhanced Locality Learning with State Space Models for Video Deraining rely on a 3D Hilbert scanning mechanism to minimize the dilation factor and prevent the loss of local structural information.

2. Coherent Context for Autoregressive Transformers

For generative models that predict the next token based on historical context, SFCs ensure that the immediate "history" contains the spatial neighborhood of the target token. In PointGPT: Auto-regressively Generative Pre-training from Point Clouds, Guangyan Chen et al. partition point clouds into patches and order them using a Morton curve, enabling standard autoregressive transformers to learn complex geometric structures. Si-Tong Wei et al. apply a similar logic in OctGPT: Octree-based Multiscale Autoregressive Models for 3D Shape Generation, flattening a multiscale octree via Morton order to capture local geometric patterns effectively.

3. Reusing 2D and 1D Architectures

By flattening 3D data without destroying neighborhood context, SFCs allow researchers to deploy highly optimized 2D or 1D architectures directly on 3D tasks. Wanli Chen et al. introduce this capability in Efficient Point Cloud Analysis Using Hilbert Curve, flattening 3D voxels into 2D structures so that standard, computationally efficient 2D convolutional layers can process 3D geometry. For vision transformers, Da Li et al. show in Adapt PointFormer: 3D Point Cloud Analysis via Adapting 2D Visual Transformers that Morton-order serialization allows pre-trained 2D ViTs to process unordered point sets while maintaining geometric relationships.

Overcoming the Multi-Dimensional Collapse

While space-filling curves are highly effective, a fundamental mathematical limitation exists: it is impossible to perfectly map three dimensions down to one without losing some proximity information. Any single 1D curve will inevitably introduce artificial boundaries or "jumps" where points that are adjacent in 3D space are separated in the 1D sequence.

To overcome this limitation, modern architectures employ two primary strategies: multi-path scanning and domain-prior prioritization.

Multi-Path and Stochastic Scanning

Rather than relying on a single serialization path, models can scan the data from multiple distinct perspectives across different layers or heads.

Domain-Prior Prioritization

Alternatively, the serialization path can be modified to prioritize specific spatial dimensions that contain the most critical structural information.

In OccMamba: Semantic Occupancy Prediction with State Space Models, Heng Li et al. introduce a height-prioritized 2D Hilbert expansion. For outdoor scene understanding, vertical consistency (columns of objects) is highly informative. The architecture groups voxels into vertical columns first, and then applies a 2D Hilbert curve across the horizontal plane. This hybrid approach ensures that vertical structural integrity is perfectly preserved while horizontal relationships maintain high locality.

Key Foundational Work

Shape Generation using Spatially Partitioned Point Clouds (2017) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_space_filling_curve_serialization_3bf29a7b,
  title = {space-filling curve serialization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/space-filling-curve-serialization/rdr_3bf29a7bb2d54a3dab7dbc33f9c0de57},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_space_filling_curve_serialization_3bf29a7b_v1,
  title = {space-filling curve serialization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/space-filling-curve-serialization/rdr_3bf29a7bb2d54a3dab7dbc33f9c0de57/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake